The evaluation of the quality of care delivered to patients with the congenital cardiac disease relies heavily on the analysis of outcomes. Simple comparisons of overall mortality rates are not useful because of baseline differences in risk among individuals. Risk adjustment in congenital cardiac surgery is hard because of great variations in anatomy and physiology. For paediatric cardiac surgery, case mix differences between institutions lie in the anatomic variation of the underlying disease and thus in the nature of the surgical procedure required for correction. Lesion specific approaches have limited precision because even common lesions represent only a small fraction of the annual surgical caseload of a centre.
Despite everything, there is an ongoing effort to improve the techniques and technologies for evaluating the outcomes of congenital cardiac disease treatments. The rationale for this goal is multi-factorial. The importance of the quantitation of complexity centres on the basis that, in the field of paediatric cardiac surgery, analysis of outcomes using raw measurements of mortality, without adjustment for complexity is inadequate. The mix of cases can vary greatly from programme to programme. Without stratification of complexity, the analysis of outcomes will be flawed. For individual centres, quality improvement efforts can be stimulated with information on how one centre is performing compared with other centres. Evaluations of performance are important because of constant self-evaluation, including benchmarking with similar programmes and creating healthcare improvement strategies.
To create a meaningful outcome analysis system four items are necessary: a common language or nomenclature, a mechanism for data collection or a database with an established minimal database dataset, a mechanism for evaluating case complexity, and a mechanism for verifying data completeness and accuracy.Reference Jacobs, Wernovsky and Elliott1–Reference Mavroudis and Jacobs3
The common language or nomenclature has been created by the International Working Group for Mapping and Coding of Nomenclatures for Paediatric and Congenital Cardiac Disease, also known as the Nomenclature Working Group. By 2005, the Nomenclature Working Group cross-mapped the nomenclature of the International Congenital Cardiac Surgery Nomenclature and Database Project of the European Paediatric Cardiac Code of the Association for European Paediatric Cardiology, and thus created the International Paediatric and Congenital Cardiac Code.Reference Jacobs, Mavroudis and Jacobs4, Reference Jacobs, Jacobs and Mavroudis5
Two major multi-institutional efforts that have attempted to measure the complexity of congenital cardiac surgery are the risk adjustment in congenital cardiac surgery-1 method and the Aristotle basic complexity score. The methodological details of each system are described in respective references.Reference Jenkins, Gauvreau, Newburger, Spray, Moller and Iezzoni6, Reference Lacour-Gayet, Clarke and Jacobs7 In brief, the Aristotle committee, consisting of experts from 50 centres in 23 countries, developed the Aristotle basic complexity score. Each of 145 procedures was rated with a score of 0.5–5 based on these three areas of potential for mortality, potential for morbidity and technical difficulty. After assigning each procedure with an Aristotle basic complexity score of 1.5–15, each procedure was assigned to the Aristotle basic complexity level of 1–4 based on the Aristotle basic complexity score – basic score of 1.5–5.9 or basic level of 1, basic score of 6.0–7.9 or basic level of 2, basic score of 8.0–9.9 or basic level of 3, and basic score of 10.0–15.0 or basic level of 4. In contrast, the risk adjustment in congenital cardiac surgery system, which was developed between 1993 and 1995, congenital cardiac operations were stratified into one to six categories. The Aristotle basic complexity score and risk adjustment in congenital cardiac surgery method represent the most widely used and accepted methodologies of case mix complexity adjustment in paediatric and congenital cardiac surgery. Both tools for stratification of complexity are slightly different and each is only an approximation of stratification of complexity and not proper risk adjustment. With both systems, as the complexity increases, mortality before discharge from the hospital also increases. Efforts involving the developers of each system to unify these two systems so as to capitalise on the strengths of both are already underway.Reference Jacobs, Jacobs, Jenkins, Gauvreau, Clarke and Lacour-Gayet8
In this report, we analysed outcomes of cardiac surgical procedures performed on population with congenital cardiac defect born between October 1, 2002 and October 1, 2007. Data included in this report were collected in this period of study and represent surgical procedures performed with or without cardiopulmonary bypass. This section contains information on patient, discharge mortality, occurrence of postoperative complications, and procedural complexity. Case complexity is presented using both the Aristotle basic complexity score and the risk adjustment for congenital cardiac surgery methodology. An account of the fact that improvements in the medical care of patients with congenital cardiac disease, including early diagnoses, diagnostic procedures, neonatal care and intensive care units in our country, were not followed by an advance in surgical development, major proportion of cardiac surgeries is still performed in referral centres abroad. By employing the widely used and accepted methodologies of case mix, complexity adjustment in paediatric and congenital cardiac surgery, we tried to evaluate the performance in our surgical centres, as well as the paediatric cardiac surgery programme in our country.
Materials and methods
The study population consisted of all patients with congenital cardiac disease born to Croatian residents between October 1, 2002 and October 1, 2007 undergoing a cardiovascular operation during this period. Information about the patients was obtained from medical records collected by a paediatrician cardiologist. Data were collected from the Croatian database, the base that had been created as population and hospital-based register and which included all children with congenital cardiac disease born between October 1, 2002 and October 1, 2007, a year with demographic and in-hospital variables (Table 1). A resident was considered to be anyone who had lived in Croatia for a year or longer. All diagnoses and procedures were coded according the International Paediatric and Congenital Cardiac Code. Cardiac surgical procedures performed with or without cardiopulmonary bypass are also included in the report. The primary procedure for an operation is the procedure with the highest Aristotle basic complexity score with two exceptions: in the event of simultaneous bidirectional Glenn and pulmonary artery arterioplasty, the bidirectional Glenn is the primary procedure and in the event of a primary diagnosis of atrial septal defect, sinus venosus with procedures of partial anomalous venous return repair and atrial septal defect repair, patch, the second will be considered the primary procedure. If the two procedures within a given operation share the highest Aristotle basic complexity score, then the procedure designated as primary by the patient will be used.
Patient characteristics according to the age at time of operation and presence of other non-cardiac anomalies.
The mortality is achieved using a patient admission-based mortality calculation with a numerator of the number of deceased patients and a denominator of the number of cardiac surgical patient admissions. Mortality that occurs for an admission with multiple operations is assigned to the first cardiac operation, that is, the first operation with operation type cardiovascular bypass or no cardiovascular bypass cardiovascular of that admission.
This initial cardiac operation of the hospitalisation is considered as the index operation of the hospitalisation. Mortality status at discharge is the chosen measure of mortality for this report. Patients weighing less than or equal to 2500 gram undergoing patent arterial duct ligation as their primary procedure will not be included in the mortality calculation because of the fact thatmost of the deaths in this patient population are multi-factorial and largely unrelated to the surgical procedure in time and by cause.Reference Bové, Francois and De Groote9 Case mix complexity adjustment is performed by using both the Aristotle basic complexity score and the risk adjustment for congenital cardiac surgery methodology. The Aristotle basic complexity score, Aristotle basic complexity level and risk adjustment in congenital cardiac surgery method-1 are calculated at the patient admission level. For each patient cardiac admission, the Aristotle basic complexity score and level and risk adjustment in congenital cardiac surgery categories are determined by the component procedure of highest complexity for the index operation. After cardiosurgical procedure were included and measured in this report, 10 complications were reported of unplanned reoperation, cardiac arrest, atrioventricular block requiring permanent pacemaker, sternum left open, acute renal failure requiring dialysis, reoperation for bleeding, mediastinitis, neurological deficit persisting at discharge, new onset seizures, and prolonged stay in intensive care unit. The postoperative complications were presented as prolonged length of stay of more than 21 days.
Statistical methods
In spite of relevant findings, we primarily used descriptive statistics. Difference in average complexity for cardiac procedures performed in Croatia and the ones performed in abroad according to Aristotle basic complexity score and risk adjustment in congenital cardiac surgery method was tested by non-parametric Mann–Whitney U test. Association of mortality and registration of postoperative complications with the place of operation was tested with Fisher’s exact test. Association of Aristotle basic complexity score and risk adjustment method distribution with mortality and postoperative complication was tested with Pearson’s χ2 test, and the degree of association between variables was assessed with Cramer’s V coefficient. All analyses were conducted using statistical package for social sciences version 13.
Results
Of the 556 operations, the Aristotle complexity basic score could be assigned to 553 operation and risk adjustment in congenital cardiac surgery method-1 score to 536 operations. Operations that were assigned by the Aristotle complexity basic score or risk adjustment method and with known discharge mortality status were used in all subsequent analysis. Among 553 cardiac surgeries performed which could be scored by the Aristotle complexity basic score, 202 surgeries were performed in Croatia and 351 surgeries were performed in abroad cardiosurgical centres, among 536 procedures classified by risk adjustment in congenital cardiac surgery-1 methodology, 336 surgeries were performed abroad and 200 surgeries in Croatia. Procedures were performed in two institutions in Croatia and seven institutions abroad, cardiosurgical centres across the west Europe. The overall mortality rate among procedures analysed by the Aristotle complexity basic score was 4.3%, with the rate of complications reported after cardiosurgical procedure being 24.1%. The overall mortality rate among procedures classified by risk adjustment in congenital cardiac surgery-1 methodology was 4.5%, with the rate of complications reported after cardiosurgical procedure being 24.3% (Table 2). About 84.1% of procedures were performed before the age of 1 year. The average complexity, according to the Aristotle complexity basic score for cardiac procedures performed in Croatia is 6.1 and those performed abroad was 9.2 with statistically significant difference. The average complexity, according to the risk adjustment methodology for cardiac procedures performed in Croatia is 2.2 and those performed in abroad is 3.1 with statistically significant difference. In Croatia, only five procedures in risk adjustment in congenital cardiac surgery-1 method category four were performed, and no cases in category five or six. Also, there were only four procedures placed in the Aristotle basic level 4 performed in Croatia (Table 3). Distribution of cases operated abroad by complexity information, mortality rate, and occurrence of postoperative complications is given in Table 4.
Distribution of all cases by complexity information, mortality rate, and occurrence of postoperative complications (PLOS).
PLOS, prolonged length of stay; RACHS, risk adjustment in congenital heart surgery
Distribution of cases operated in Croatia by complexity information, mortality, and occurrence of postoperative complications (PLOS).
PLOS, prolonged length of stay; RACHS, risk adjustment in congenital heart surgery
Distribution of cases operated abroad by complexity information, mortality rate, and occurrence of postoperative complications (PLOS).
PLOS, prolonged length of stay; RACHS, risk adjustment in congenital heart surgery
The most frequent procedure performed in Croatia was pulmonary artery banding, followed by ventricular septal defect repair, coarctation repair, patent arterial duct closure, modified Blalock Tausig shunt, and atrial septal defect repair. The most frequent procedure performed in abroad was arterial switch followed by Norwood procedure, bidirectional Glenn, atrioventricular canal repair, tetralogy of Fallot repair, and double outlet right ventricle repair (Table 5). Among 202 procedures in Croatia classified by the Aristotle complexity basic score, or 200 procedures by risk adjustment in congenital cardiac surgery-1 methodology, death occurred after 10 procedures with mortality rate of 5%, on the other hand, among 351 procedures performed abroad and assigned by the Aristotle complexity basic score, death occurred after 14 procedures with calculated mortality rate of 4%. After 336 procedures performed in abroad and classified by risk adjustment in congenital cardiac surgery-1 methodology, death occurred after 14 procedures with mortality rate of 4.2%. There is no significant difference between mortality in Croatia and abroad without adjustment for complexity. Among 202 procedures performed in Croatia and which could be classified by the Aristotle complexity basic score, or 200 procedures according to risk adjustment in congenital cardiac surgery-1 methodology, postoperative complications were registered after 50 procedures with 24.8% or 25%. In contrast, among 351 procedures performed in abroad, there were postoperative complications after 83 procedures with 23.6% or by risk adjustment in congenital cardiac surgery-1 methodology, among 336 procedures, postoperative complications were registered after 80 procedures with 23.8%. There is no significant difference, between registration of postoperative complications in Croatia and abroad, without the adjustment for complexity.
The top six (by frequency) primary procedures, in Croatia and in abroad, with incidence, discharge mortality, and complexity.
ABC, Aristotle basic complexity score; RACHS 1, risk adjustment in congenital heart surgery
Among procedures performed in Croatia and abroad, considerable variation in mortality and postoperative complications was observed in the Aristotle basic level 2 and 3 with significantly higher mortality of procedures performed in Croatia in the Aristotle basic level 2, with a mortality rate of 4.8 in Croatia in relation to 1.9 in abroad, and in level 3 with mortality rate 13% in relation to mortality among procedures performed in abroad of 1.7% (Tables 3 and 4 and Fig 1). By using classification according to risk adjustment in congenital cardiac surgery-1 methodology, significantly higher mortality rate was observed in group 2 with mortality rate among procedures performed in Croatia being 7.4% in relation to 1% performed in abroad (Tables 3 and 4 and Fig 1). When analysing the differences in occurrence of postoperative complications, there is significantly higher rate of postoperative complications in the Aristotle basic level 3, with a rate of 34.8% among procedures performed in Croatia in relation to 15% among procedures performed in abroad, and in group 3 according to the risk adjustement in congenital cardiac surgery-1 methodology, with 36.2% of postoperative complications among procedures performed in Croatia in relation to 22.4% among procedures performed in abroad (Tables 3 and 4 and Fig 2). When analysing all procedures assigned by the Aristotle complexity basic score, we determined no statistically significant connection between the Aristotle complexity basic score and mortality rate, but we evinced that there is statistically significant connection between the Aristotle complexity basic score and mortality only after exclusion of procedures performed in Croatia. When analysing all procedures assigned by the Aristotle complexity basic score, we revealed statistically significant connection between the Aristotle complexity basic score and postoperative complications and this connection is stronger only when the analysing procedures were performed in abroad. When studying all procedures that could be classified by the risk adjustement methodology in congenital cardiac surgery-1, we determined no statistically significant connection between Risk Adjustement in congenital cardiac surgery-1 methodology groups and mortality rate, or when we studying procedures done only abroad. This finding might be because of the small number of deaths occurred in the groups. By analysing the relation between risk adjustment in congenital cardiac surgery-1 methodology groups and percentage of postoperative complications, when studying all procedures, we determined a statistically significant connection and the association is stronger only when analysing procedures were performed in abroad.
Relationship between the Aristotle basic complexity level/risk adjustment in congenital heart surgery-1 categories and observed mortality of all cases, among procedures performed in abroad and in Croatia.
Relationship between the Aristotle basic complexity level/risk adjustment in congenital heart surgery-1 categories and observed occurrence of postoperative complications of all cases, among procedures performed in abroad and in Croatia.
Discussion
There is huge variety of unadjusted mortality rates between centres performing operations of congenital cardiac disease, with a mortality rate ranging 2.5–11.4%.Reference Jenkins and Gauvreau10 Earlier reports illustrated well that an unadjusted mortality rate is inadequate for evaluating institution quality and that simple statistics can be misleading. Both the Aristotle basic complexity score and risk adjustment in congenital cardiac surgery-1 methodology were predictive of in-hospital mortality as well as of prolonged postoperative length of stay, but both tools for stratification of complexity are slightly different, each is only an approximation of stratification of complexity, not true risk adjustment and both might have several weaknesses. The Aristotle methodology allows classification of more operations, while the risk adjustment in congenital cardiac surgery-1 system discriminates better at the higher end of the complexity.Reference Jacobs, Mavroudis and Jacobs11 Neither the risk adjustment in congenital cardiac surgery-1 system nor the Aristotle basic complexity score incorporate detailed patient specific risk factors into their algorithms.Reference Jacobs, Wernovsky and Elliott1 In contrast, the risk adjustment in congenital cardiac surgery method has been showed to be a useful tool in several studies in both Europe and North America and represents one of the first widely accepted tool for adjustment of complexity developed in the field of paediatric cardiology.Reference Larsen, Pedersen, Jacobsen, Johnsen, Hansen and Hjortdal12, Reference Boethig, Jenkins, Hecker, Thies and Breymann13 The database of Society of Thoracic Surgeons and the European Association for Cardio Thoracic surgery have been including the Aristotle complexity score in their reports since 2002.Reference Jacobs, Jacobs, Mavroudis, Lacour-Gayet and Tchervenkov14 The European Association for Cardio Thoracic Surgery and the Society of Thoracic Surgeons multi-institutional database indicate that the Aristotle complexity score correlates well to mortality prior to discharge from the hospital after congenital cardiac surgery, as well as to prolonged postoperative stay.Reference O’Brien, Jacobs and Clarke15 Although, some papers favour one over other,Reference Al-Radi, Harrell and Caldarone16 in 2006, both databases, the databases of Society of Thoracic Surgeons and The European Association for Cardio Thoracic Surgery, unified two adjustment systems and incorporated the Aristotle complexity basic score and risk adjustment in congenital cardiac surgery-1 methodology into their reports.
In the databases of both, the Society of Thoracic Surgeons and European Association for Cardio Thoracic Surgery, mortality before discharge is now between 4% and 5%.Reference Jacobs, Jacobs and Franklin2
Here we reported that the mortality rate in Croatia was 5%, and, on the other hand, the mortality rate in procedures performed in centres that helped in treating our patients was 4%. As this report illustrated there are marked differences in mortality and occurrence of postoperative complications between Croatia and centres abroad only after adjustment for complexity. By using statistical analysis to test the association between the Aristotle complexity basic score or risk adjustment in congenital cardiac surgery-1 methodology groups with mortality or morbidity rate, we determined association, or we find out stronger association after exclusion of procedures performed in Croatia, by analysing only procedures performed abroad. In our case, this strongly suggests relevance of institution where a procedure is performed and indicates to a necessity of readjusting performance in our centres. In some risk categories there is significantly higher mortality and morbidity after procedures performed in Croatia. This may be explained by many reasons. Centres in Croatia could be classified as smaller centres performing less than 200 procedures per year, with a mortality of about 4–5% only for selected cases, with more complex patients being sent away. Paediatric cardiac surgery is the only surgical procedure performed in children for which a volume outcome relationship has been documented. Earlier reports showed that unadjusted mortality rate in very small hospitals was no different than at large hospitals or might be even lower than in medium volume hospital. After adjustment for complexity large hospitals performed significantly better than all other volume groups.Reference Gauvreau17 However, large volume hospitals performed more complex operations and achieved superior results. Although, a great proportion of published studies reported a statistically significant relationship between higher institutional case volume and better clinical outcomes, the most recent study suggested that this relationship might no longer exist and illustrated the inappropriateness of using volume alone as a marker of quality.Reference Welke, Diggs, Karamlou and Ungerleider18 When looking at mortality of those cases, having in mind that these cases are mostly classified in lower risk groups, mortality is higher than in reports from some other centres, suggesting clearly that there is a need to support and organise a retraining of our programme. As congenital cardiac defects in our country, still sometimes operated by surgeons who operate on adult patients, mortality rate, especially in some categories, could be higher as indicated by the report about procedures performed on adult patients.Reference Karamlou, Diggs, Person, Ungerleider and Welke19
Although outcome analysis is potentially dangerous with many implications, centres should not fear the potentially negative consequences of reporting less than stellar results. The aim is to identify the problems and institute improvement initiatives, which can include inter-institutional team visits, mentoring schemes, and educational programmes. These kinds of inter-institutional visits have succeeded in improving outcomes in most countries.Reference Larrazabal, Jenkins and Gauvreau20 As reported by Novick et al,Reference Novick, Stidham and Karl21 Croatia was included in such a programme from 1993 to 2003. After these years, the challenges to improve a standing programme in paediatric cardiac surgery in Croatia still exist. The results of this study might have important implications in the need for centralisation in paediatric cardiac surgery to achieve best possible results is once again showed, Europe still does not have a well established pathway for international cooperation in the field of healthcare and remains highly dependent on national structures, this study highlights how international cooperation is highly beneficial to provide best practice care for a particular patient population and in particular when it comes to rare diseases in small countries, the use of standardised risk scores allows selection of international partners according to evidence based outcomes for specific defects which may have very different outcomes in various centres, and finally, the results show how risk adjusted outcomes may be used to develop initiatives, to improve the services provided on a national level and also to develop cooperation between neighbouring small countries.
However, hospital discharge data and mortality rate may not be the only or the best way to evaluate congenital cardiac surgery program. Despite its importance, in-hospital mortality is only one of the clinically important end points that should be of interest to program directors. Relative performance with respect to postoperative complications, prolonged length of hospital stay in intensive care unit, late mortality and functional and neurological outcomes would also be of obvious interest for efforts at quality improvement. The databases of the Society of Thoracic Surgeons and European Association for Cardio Thoracic Surgery currently do not allow for long-term follow-up, and analysis of outcomes must reach beyond mortality, and encompass longer term follow-up, including cardiac and non-cardiac morbidities and impacting health-related quality of life and including functional state through the classification of the New York Heart Association.Reference Jacobs, Wernovsky and Elliott1, Reference Jacobs, Mavroudis and Jacobs4, Reference Clarke, Lacour-Gayet and Jacobs22
