Congenital interventional cardiologists in the United Kingdom and Ireland have no effective method of collating, benchmarking and acting on morbidity and mortality data from the catheterisation laboratory. The ways in which National Institute for Cardiovascular Outcomes Research collates and handles submitted outcome data do not allow for extensive exploration of morbidity data to facilitate quality improvement. Reference Gale, Weston and Denaxas1 Therefore, the prospective assessment of risk with respect to congenital catheterisation is limited and must be accounted for by cardiologists who assimilate the potential risks based on arbitrary clinical data and experience as they prepare to counsel a family and plan for a case. This is in striking comparison to other specialties whose regulatory bodies have established tools to quantify risk prior to anaesthesia or invasive procedures, such as The American Society of Anesthesiologists Physical Status Classification System Reference Daabiss2 and the Aristotle Score in Congenital Cardiac Surgery. Reference Lacour-Gayet, Clarke and Jacobs3 Congenital interventional cardiology in the United Kingdom and Ireland is still seeking a unified methodology for risk prediction and benchmarking.
In the United States of America, two specific quality assessment tools have been developed using retrospective data from several thousand cardiac catheterisations, mainly from within the United States of America. These are routinely used by several institutions around the United States of America. They are the Catheterization for Congenital Heart Disease Adjustment for Risk Method score, developed by the Congenital Cardiac Catheterisation Project on Outcomes group Reference Learn, Holzer and Daniels4–Reference Bergersen, Gauvreau and Foerster6 and the Congenital Cardiac Interventional Study Consortium derived Catheterisation RISk score for Paediatrics. Reference Nykanen, Forbes and Du7 The Catheterisation RISk score for Paediatrics score, designed for use in paediatric congenital cardiac catheterisation cases, encompasses eight input parameters calculating a “CRISP score.” This score determines a “CRISP risk category” numbered 1–5 which is associated with a percentage risk for the planned procedure. The inputs and outputs are seen in Table 1. Reference Nykanen, Forbes and Du7 The pre-procedural input parameters represent routine clinical data available to all physicians undertaking the care of children and/or adults with congenital heart disease.
Allocation of scores for CRISP variables by category.
CRISP categories: 1 = CRISP score 0–2 (1% AER), 2 = CRISP score 3–5 (2.6% AER), 3 = CRISP score 6–9 (6.2% AER), 4 = CRISP score 10–14 (14.4% AER), 5 = CRISP score 15+ (36.8% AER)
Adapted Nykanen et al. Reference Nykanen, Forbes and Du7
AER = adverse event rates; CRISP = catheterisation RISk score for paediatrics; ECMO = extracorporeal membrane oxygenation
* Characteristics marked are predefined in the original article which describes and defines the CRISP score Reference Nykanen, Forbes and Du7
The United Kingdom and United States of America are somewhat discrepant in their provision of medical care based on differing models of access to care and as well as resource availability amongst other determinants. Reference Gilboa Suzanne, Devine Owen and Kucik James8–Reference Dolk, Loane and Garne11 The application of quality measures and risk assessment based on data accounting for US practice may not be immediately transferable to UK practice without validation. Establishing whether parallels can be drawn and validating the Catheterisation RISk score for Paediatrics against current benchmarks for adverse event reporting may provide an opportunity to employ the Catheterisation RISk score for Paediatrics during the counselling and consent process. It may facilitate more focused procedural planning prior to catheterisation. Lastly, Catheterisation RISk score for Paediatrics could provide a platform for comparison with international centres of excellence and may facilitate service development and interinstitutional collaboration in cardiac catheterisation across the United Kingdom and Ireland. The aim of this study is to ascertain whether the Catheterisation RISk score for Paediatrics assessment tool accurately predicts the likelihood of significant morbidity and mortality within United Kingdom and Irish paediatric and adult congenital cardiac catheterisation practice.
Materials and methods
All major congenital cardiac units across the United Kingdom and Ireland were invited to contribute to this study. Of these, five participated. Three of these provide paediatric and adult congenital cardiac catheterisation in the same institution, whereas the other two provide paediatric cardiac catheterisation services only. The centres cover a wide geographical area throughout the British Isles and deal with a high level of complexity with respect to congenital catheterisation and surgical care. The case inclusion criteria were for all diagnostic and interventional congenital cardiac catheterisation cases. All invasive electrophysiological assessments, as well as airway interventional procedures were excluded, as these are outside the scope of the Catheterisation RISk score for Paediatrics scoring tool.
The average annual case numbers from all United Kingdom and Irish centres over a 5-year period from 2011 to 2016 is approximately 300 cases. 12 Therefore, we sought to retrospectively collect data on 300 consecutive cases from each centre. This provided a sample size of 1500 congenital cardiac catheterisation procedures. Although no formal sample size power determination was performed, this was agreed by the investigators as being a representative sample.
After the agreement of local institutional research review boards, an independent research fellow travelled to each institution between October 2016 and March 2017 and accessed and reviewed in-house prospectively recorded data from local databases and patient medical records. All independent variables used to qualify a Catheterisation RISk score for Paediatrics score were recorded using a secure limited data collection tool. Additional demographic data were collected. A Catheterisation RISk score for Paediatrics category and predicted adverse event rate (%) were documented for each case. Observed catheterisation-related adverse events were then recorded and categorised according to the catheterisation complication classification system defined by the UK National Institute for Cardiovascular Outcomes Research (Table 2). 12 This classifies adverse events as mild, moderate, major and catastrophic. The observed incidence of complications in each National Institute for Cardiovascular Outcomes Research category was then compared with the Catheterisation RISk score for Paediatrics category to assess the potential for Catheterisation RISk score for Paediatrics to predict risk in our patient cohort. Mild adverse events as defined by National Institute for Cardiovascular Outcomes Research were excluded from analysis. Such adverse events may be vague, reversible, reported discrepantly and are less likely to alter a patient’s course. CRISP is designed to predict serious adverse events which can be mapped to National Institute for Cardiovascular Outcomes Research complications classifications of at least moderate severity. This level of adverse event usually results in a significantly altered clinical course for the patient, even in the event of complete resolution. Statistical analysis was performed using SPSS v24 (IBM Corp). Categorical data are presented as frequencies and percentages. Continuous data are presented as medians with ranges or means with a standard deviation based on the reported variable. Association was analysed using Chi-square analysis with an alpha of 0.05.
NICOR congenital cardiac catheterisation adverse event severity classification system. 12
The moderate to catastrophic categories correlate well with the definitions of significant adverse events; incidences of which formed the basis for the CRISP score
Results
Patient records and reported data from a total of 1500 congenital cardiac catheterisation procedures were reviewed across five major congenital cardiac centres, from procedures spanning an overall time period of August 2015–October 2016 inclusive. The timeframe for 300 consecutive eligible cases to be completed by each individual centre was comparable and is demonstrated in Figure 1. Review of all collected cases revealed 1483 (98.9%) unique cases where adequate data was available to allow a retrospective calculation of a Catheterisation RISk score for Paediatrics score, as well as to facilitate the recording and classification of adverse events, as defined by National Institute for Cardiovascular Outcomes Research, for later comparison between expected and observed complications. Centres A, C and E performed both adult and paediatric cardiac catheterisations.
Data collection timeline over which 300 cases were performed in each of five centres. The range over which the specified number of procedures (n = 300) were performed was quite uniform with a range of 11–16 months.
Baseline and procedural data
From the 1500 cases scrutinised, 17 (1.1%) were excluded after blinded review by the investigators who determined those data records were afflicted by discrepancies which significantly affected the veracity of the Catheterisation RISk score for Paediatrics calculation. Of the remaining 1483 cases for analysis, 1130 (76.2%) catheterisations were performed in paediatric patients across the five centres. A total of 353 (23.5%) adult congenital heart disease catheterisations were performed in the three centres treating adult patients. The number of adult congenital heart disease interventions was similar between these three centres (n = 119, 101, 133).
The mean age of paediatric (<18 years) patients was 4.2 years (±4.8). The mean was similar across all centres. The mean age of adult patients (>18 years) across three centres was 39.8 years (SD ± 15.8) and was similar across these centres (40.6, 40.2, 38.7 years).
Hybrid interventions accounted for 2% of cases, 29% were diagnostic evaluations and 69% of procedures were non-hybrid interventional catheterisations. The distribution of diagnostic, interventional, and hybrid interventions combined for all centres for both paediatric and adult congenital heart disease interventions (B) as well as the distribution of routine and urgent cases are displayed in Figure 2. The range of incidence of diagnostic catheterisations across centres varied from 16 to 41%. The range of incidence of interventional catheterisations across all centres ranged from 58 to 79%. Three centres performed hybrid interventions during the study period.
Case distribution by urgency ( a ) and case type ( b ) across the total population and characterised by adult or paediatric population.
The majority of cases were classified as elective (85%), with the remainder (15%) either classified as emergent (within 24 hours) or urgent (within 1 week). There was no statistically significant difference in these characteristics between adults and children. Centre B classified catheterisation urgency as either emergent or elective. The remaining four centres classified their cases as elective, urgent and emergent. In centres performing both paediatric and adult congenital interventions, urgent and emergent catheterisation were more commonly required in paediatric patients.
In the 1130 paediatric cases, 44 out of 924 elective (4.8%), 19 out of 97 urgent (19.6%) and 12 out of 109 emergency (11.0%) catheterisations were associated with a significant complication. Of 353 adult congenital heart disease cases, 19 out of 336 elective (5.7%), 1 out of 15 urgent (6.7%) and 0 out of 2 (0%) emergency cases had a significant catheterisation-related complication.
Predicted complications (CRISP)
The most common Catheterisation RISk score for Paediatrics category in this study was CRISP 2 (41.3%) and the least common was CRISP 5 (1.3%). In paediatric patients, the most common Catheterisation RISk score for Paediatrics category was CRISP 2 in three centres and CRISP 3 in two centres. In adults, CRISP 2 was the most common category. The range of Catheterisation RISk score for Paediatrics was 0–19 with a median score of 5 across all centres (Fig 3). The was no significant difference in Catheterisation RISk score for Paediatrics distribution between centres performing paediatric only and mixed paediatric and adult congenital practice however, there was a significant difference in Catheterisation RISk score for Paediatrics distribution between paediatric and adult cases (p < 0.0001) (Table B, Appendix). The remaining Catheterisation RISk score for Paediatrics category data is available in the appendix (Table A, Appendix).
Box and whisker plot of CRISP scores (1–19) demonstrating the distribution of scores by centre as well as any outlying data that did not contribute to the mean or IQR (*, n=). No statistically significant difference in median CRISP risk scoring was seen between centres. IQR = interquartile range.
Observed complications (NICOR classification)
According to the National Institute for Cardiovascular Outcomes Research classification of complications, 1285 patients (86.6%) in this study did not suffer a complication. Mild complications which are not accounted for in the Catheterisation RISk score for Paediatrics prediction model were noted in 100 patients (6.7%). The range of mild complications by centre was 3−9.9%.
Ninety-eight (6.6%) patients in this study experienced a significant complication (moderate, severe or catastrophic). Fifty-nine (4%) experienced a moderate complication, 34 (2.3%) experienced a major complication and 5 (0.3%) experienced a catastrophic complication resulting in death.
Seventy-seven (79%) significant complications were seen in paediatric patients. Of these, 47 (61%) were moderate, 26 (33.8%) were major and 4 (5.2%) were catastrophic resulting in death. The remaining 21 patients were adult congenital heart disease patients and in this group, the majority of complications were moderate (n = 12; 57.1%). Eight of the complications were severe (38.1%) and 1 was catastrophic (4.8%). The incidence of complications by severity, centre and where relevant, patient group can be seen in the Appendix Table C.
There was no statistically significant difference in complication rates by category between adults and paediatric patients, nor between centres performing mixed practice versus paediatrics alone (Table D, Appendix).
Comparative adverse event data: observed (NICOR) and predicted (CRISP)
Of the 202 patients who were classified as CRISP 1, 3 patients (1.5%) suffered complications. Two moderate and one a major complication. One complication was in a paediatric patient and two in adult patients. There was no statistically significant difference between observed adverse events and predicted complication rates in CRISP 1 patients suggesting that patients classified in CRISP 1 are accurately categorised based on National Institute for Cardiovascular Outcomes Research complication definitions (Fig 4, Table 3).
Bar chart by CRISP category demonstrating the observed adverse event rate in paediatric, adult and combined cases where the flat line traversing the four bars represents the predicted adverse event rate by CRISP.
Table demonstrating the frequency of both observed and predicted (expected) AER’s for each CRISP category in both paediatrics and adults.
Percentage difference, Chi-square analysis, and significance are qualified for each group
X2=chi squared; n/a=no available data for analysis
Similar trends were noted for CRISP 2 and 3 patients where there was no significant difference noted between observed and predicted adverse event rates. This holds true for both paediatric and adult patients when accounted for independently (Fig 4, Table 3).
In CRISP 4, there was a significant difference between predicted and observed adverse event rates in adult patients (p < 0.0001). This trend was not seen in paediatric patients (Fig 4, Table 3).
In CRISP 5, there was no significant difference between predicted and observed adverse event rates in paediatric patients (p = 0.59). There was no estimate made for adult patients as there were no data to analyse (Fig 4, Table 3).
When comparing paediatric and adult cases, there was no significant difference in adverse event rates by Catheterisation RISk score for Paediatrics category (p = 0.521). When comparing centres performing paediatric only cases to centres performing mixed paediatric and adult cases, there was a near significant difference in adverse event rates based across Catheterisation RISk score for Paediatrics categories (p = 0.064; Table E, Appendix).
Discussion
Congenital cardiac catheterisation practice varies between operators, institutions and health services. The individualisation of care, innovative practices and increasing complexity of those surviving with congenital heart disease means standardising practice for the purposes of risk assessment and comparison is challenging and ever changing. Patient, procedural and operator factors all have a potential influence on outcomes for any given intervention. Despite discussions on this topic for decades, Reference Phillips, Cabalka, Hagler, Bailey and Cetta13–Reference Bergersen, Gauvreau, Jenkins and Lock15 an all-encompassing risk assessment tool has been challenging to develop, given the heterogeneity in practice and patients, in a specialty where adverse event rates are relatively low. In the United States of America, the Catheterization for Congenital Heart Disease Adjustment for Risk Method and Catheterisation RISk score for Paediatrics have been developed using mostly North American data and are being used at least qualitatively in routine practice in many North American institutions. Reference Bergersen, Gauvreau and Foerster6,Reference Nykanen, Forbes and Du7
Our data demonstrate aspects of practice variation across congenital cardiac centres within the same country and health service. There is variable co-location of adult congenital heart services with paediatric cardiac services in the United Kingdom and Ireland. This means that each centre has different proportions of adult congenital practice, a patient group whose risk assessment involves additional and unique consideration when compared to children. Reference Stefanescu Schmidt, Armstrong, Kennedy, Nykanen, Aboulhosn and Bhatt16,Reference Taggart, W. and Forbes17 A similar discrepancy in practice is seen due to the variable availability and use of advanced non-invasive imaging, such as cardiac CT or MRI. Reference Heathfield, Hussain and Qureshi18,Reference Crean19 This leads to a variation in the number of purely diagnostic cardiac catheterisations between units, depending on their access to advanced non-invasive imaging. This will have an impact on the risk profile for a department which reserves the catheterisation laboratory for patients who require an interventional procedure, or which also undertakes more complex invasive haemodynamic assessments.
The C3PO group’s CHARM score does enable the comparison of data across institutions and practitioners but does not attempt to predict risk in respect to procedural planning and counselling. Reference Learn, Holzer and Daniels4–Reference Bergersen, Gauvreau and Foerster6 Catheterisation RISk score for Paediatrics was designed with similar intentions, but its focus is the prediction of risk prior to catheterisation in paediatric congenital heart practice. The databases upon which these risk calculators are based (CCISC and C3PO) also serve to facilitate comparison between institutions and provide the opportunity to identify areas of unmet need in practice either within or amongst institutions and healthcare systems.
The results of this study suggest that the use of the USA-derived Catheterisation RISk score for Paediatrics scoring tool in congenital cardiac catheterisation practice within the United Kingdom and Ireland is valid. Our study cohort demonstrates similar baseline characteristics to the original population used to develop CRISP. Reference Nykanen, Forbes and Du7,Reference Hill, Du and Fleming20 The UK-based NICOR classification system 12 for complications in congenital heart catheterisation has allowed us to validate CRISP and determine it as a practical, useful and accurate method of predicting risk. There was no significant difference in CRISP predicted risk and true incidence of observed complications in our paediatric patient population. Case complexity was reasonably uniform across all centres. Complications rates were comparable across centres and similar between children and adults. This demonstrates the potential applicability of this metric across centres with practice variability within similar health systems that benchmark to similar standards. It also demonstrates that these metrics may be used as a useful comparison between centres both within and between health systems and even nations. A United Kingdom or indeed European-wide specific prospective analysis could confirm reliability and applicability. The prospective introduction of this tool into routine congenital cardiac catheterisation practice across the United Kingdom and Ireland would also allow longitudinal performance benchmarking between centres as well as within centres using standardised and validated data. All UK centres currently submit data as part of a mandated annual audit through National Institute for Cardiovascular Outcomes Research and the expansion of this data collection to account for variables defined by Catheterisation RISk score for Paediatrics as part of their scoring system would be easy to implement.
The shortcomings of the Catheterisation RISk score for Paediatrics in evaluating adults with congenital heart disease are evident in this analysis and are unsurprising. The original scoring system did not include adult congenital data as part of its development. The data inputs that are included as part of CRISP are far more representative of clinical data affiliated to infants and younger children and not the ageing adult congenital heart disease patient; for example, weight and age categories discriminate in ranges applicable to neonates, infants and children (<2.5 kg, 2.5–5 kg, >5 kg; <30 days, 30 days – 1 year, >1 year). Adult patients surviving with congenital heart disease who require diagnostic re-evaluation and catheter re-intervention may be more physiologically heterogenous and sometimes complex. Those complexities may not be age or weight dependent. Although many of the complexities associated with adult congenital cardiac patients may be determined by their anatomical and surgical substrate, the age related, degenerative and lifestyle-dependent disease must be acknowledged, as well as the comorbid medical complexity associated with these patients. These may include concurrent coronary arterial and heart muscle disease, systemic hypertension, metabolic, pulmonary and renal disease. Reference Crossland, Van De Bruaene, Silversides, Hickey and Roche21–Reference Dimopoulos, Diller and Koltsida24 The targeted addition of relevant risk modifiers for the adult congenital heart disease population may allow a modified version of Catheterisation RISk score for Paediatrics to provide a more comprehensively accurate risk prediction across the complete age range of patients who present with congenital heart disease for cardiac catheterisation. This has already been explored by some congenital groups which have published their preliminary experience in the literature. Reference Learn, Holzer and Daniels4,Reference Stefanescu Schmidt, Armstrong, Kennedy, Nykanen, Aboulhosn and Bhatt16,Reference Taggart, W. and Forbes17 Based on our evaluation as part of this study, we have introduced modifiers to the Catheterisation RISk score for Paediatrics scoring tool and hope to validate and publish these.
As part of the intention to streamline risk stratification in our cardiac catheterisation laboratory at Children’s Hospital of Colorado, a pre-populatable Catheterisation RISk score for Paediatrics scoring and categorisation tool within the Electronic Medical Record pre-operative assessment has been developed and implemented. This provides alerts for cardiac anaesthetists involved with the case, blood transfusion services and bed managers in the cardiac ICU. A range of adult modifiers of disease have been developed and are included in the Electronic Medical Record, which should facilitate evaluation within and beyond the institution to determine if these metrics improve our capacity to predict risk in this highly complex group of patients. Some explanatory outputs from our study may explain the lack of correlation between Catheterisation RISk score for Paediatrics predicted risk and observed adverse events; particularly the lack of representation of adult congenital heart disease patients in the CRISP 4 and 5 categories. This may be due to the inherent biases of the Catheterisation RISk score for Paediatrics scoring tool, as previously described.
Whilst this study has not sought to analyse the physiological, diagnostic or procedural scoring system used as part of the Catheterisation RISk score for Paediatrics tool, it has been our experience in working with this dataset that there are some limitations in all three categories whereby certain diagnoses or procedures are not listed and the “closest” relevant diagnosis or procedural description must be used. Similarly, for physiological categories, ranges and figures that dictate physiological complexity, these may not be easily documented prior to catheterisation in certain patient groups and may also not represent equivalency in complexity across patients’ groups. This is probably due to the evolving complexity of the patient population but may introduce bias and discrepancies in the estimation of risk. This may manifest even more-so in the adult congenital heart disease population who may not have equivalent risk associated with a diagnosis or procedural approach as applied to a paediatric patient.
Limitations
This was a retrospective study which presents inherent bias. Although the methodology was designed to minimise sources of error, some issues with data collection must be inferred in a retrospective design. The relatively large sample size offsets this to some extent. Collection of in-house prospectively reported data and recorded medical notes retrospectively meant that missing data was inevitable and Catheterisation RISk score for Paediatrics scores were incalculable for a small number of procedures. Independent of patient complexity and case mix, cardiac catheterisation laboratories across the United Kingdom and Ireland are structured and administered in different ways. The methods of data collection and verification in each centre are unavoidably variable. Some centres routinely perform complex procedures using two consultant operators. Some centres have consultants operating independently with varying levels of experience. Some centres have senior trainees who perform catheterisations with limited supervision. Other centres are dependent on nursing staff to provide a “2nd operator” role during complex procedures. Resource availability is variable and institutional practice differs. Assessing the influence of factors like this would be more easily achieved with a validated prospective approach to data reporting and risk assessment. A prospective study and more rigorous approach to the recording of limited data within each centre to include all the required parameters for calculation of the Catheterisation RISk score for Paediatrics might provide a means of incorporating an internationally universal risk assessment tool for all congenital catheterisation laboratories operating within similar resources and patient populations.
Conclusion
The Catheterisation RISk score for Paediatrics (developed with largely North American data) accurately predicts significant complications defined by National Institute for Cardiovascular Outcomes Research in congenital catheterisation practice in the United Kingdom and Ireland. Our data validated the Catheterisation RISk score for Paediatrics assessment tool in five congenital centres through the United Kingdom and Ireland. Catheterisation RISk score for Paediatrics appears to be comprehensively transferable to United Kingdom and Irish congenital cardiac catheterisation practice for paediatric patients. Although it may be helpful and somewhat predictive of risk in adult congenital case planning, we would advise caution if using Catheterisation RISk score for Paediatrics for planning and analysis of adult congenital heart disease patients in its current format. Validating defined modifiers for adult congenital heart disease patients may result in an improved scoring system which can be used across the full age range.
Catheterisation RISk score for Paediatrics is easily implemented and requires minimal administration and staff education. Its quantification amalgamates individual patient characteristics with procedural factors which may inform patient and clinician counselling and decision making. Prospective collection of this type of data may have a long-term impact on resource allocation, education, quality assurance and even survival.
Acknowledgements
The important contribution of the technical and administrative staff involved with data access and updating of databases at each of the investigational centres and to The Bristol Heart Institute Cardiac Surgery Research Team (Massimo Caputo, Lucia Coccimello) for their statistical assistance and guidance.
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflicts of interest
None of the authors have any conflicts of interest with respect to this work.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards outlined in the Helsinki Declaration of 1975, as revised in 2008, and has been approved by the Colorado Multi-Institution Review Board (COMIRB Protocol 12-0150).
