Materials and methods

Our index was developed based upon our own experience, and the data available in the literature relating to risk factors. The reason for its development was the need to find a measure that could be predictive of the outcome in our specific Unit, and others of similar complexity. The reason for choosing each variable was our impression that surgical results are determined not only by the intra-operative procedure, but also, and particularly in developing countries, by the clinical condition of the children submitted for surgery. Our aim was to make the index simple by choosing measurements that could be obtained in most centres throughout the world, and that either had been validated previously in the medical literature, or else had frequently been observed in our own practice in relation to a poor outcome. The variables to be included were decided subsequent to meeting between the main author and senior members of the intensive care unit, who reviewed the salient literature, and the surgeons and remaining intensivists of our team.

We selected five variables to make up the index, three relating to the pre-operative period, and two to the surgical procedure itself. The variables were then stratified into categories of low, medium, and high risk, as shown in Table 1. For each variable, we allocated a score of zero for the lowest risk, a score of one for the intermediate risk, and a score of two for the highest risk, as follows:

Table 1. Variables used to compose the Index.

• Age:¹ children beyond the age of one year received a score of zero, those between the ages of one to twelve months received a score of one, and neonates received a score of two.
• Nutritional state, according to the charts from the National Centre for Health Statistics for age and sex:²⁹ children above the 10th centile received a score of zero, those between the 5th and the 10th centiles received a score of one, and a score of two was attributed to those undergoing surgery who were below the 5th centile.
• The presence or absence of the following clinical risk factors: pulmonary hypertension, cardiac failure, cyanosis, acidosis, infection, genetic syndromes, non-elective mechanical ventilation, and a pre-operative period of hospitalization in excess of one week. Pulmonary arterial hypertension was considered a risk factor when the pulmonary arterial systolic pressure was over two-thirds of the systemic pressure, as assessed non-invasively with Doppler echocardiography, independent of the degree of increased flow of blood to the lungs. Cardiac failure was considered a risk factor when it was not well controlled despite of the usage of adequate doses of diuretics and vasodilators. Severe cyanosis was defined as a peripheral arterial saturation of less than 75% at rest, and acidosis was defined as a pH less than 7.25. To be considered as a risk factor, the presence of an associated infection required radiological or laboratorial documentation, but genetic syndromes were most often diagnosed on clinical grounds. The use of mechanical ventilation prior to surgery was considered a risk factor when it was commenced because of a deterioration in the clinical condition of the patient, rather than being used as an elective measure prior to surgery. Taking note of previous experience,^{1, 11–13, 16–19} we also considered a pre-operative period of hospitalization of over one week as an associated risk factor. Patients received a score of zero in the absence of any associated clinical risk factors, scoring one when there was one associated risk factor, and two when two or more risk factors were present. All clinical risk factors were also studied independently to establish their correlation with mortality.
• Surgical complexity: we stratified our surgical procedures according to the criterions proposed both by Jenkins et al.,³⁰ and also on the basis of the Aristotle score,²⁸ calculated the index using each of them separately so as to compare results. When calculating the index using the criterions proposed by Jenkins et al., we allocated a score of zero for those procedures falling in the first and second categories, with procedures included in their third and fourth categories receiving a score of one, with the score of two applied to the procedures included in their fifth and sixth categories. As for the Aristotle criterions, we have attributed a score of zero for procedures included in their first and second categories, a score of one for the third category, and a score of two for the fourth category.
• The use and extent of cardiopulmonary bypass:^{14, 20} we allocated a score of zero when cardiopulmonary bypass was not used, a score of one when the period of bypass was up to 90 minutes, and a score of two when it was in excess of 90 minutes.

We reviewed the charts from 818 children undergoing cardiac surgery over a period of five years, from January, 2000, through December, 2004, recording the scores from the five variables into a Microsoft Excel database sheet.

All children had been admitted to a dedicated paediatric cardiac intensive care unit of a tertiary centre for paediatric cardiology, providing services for both private and national health services patients on a regional basis. The unit commenced its activities in January, 2000, and was the first dedicated unit providing paediatric cardiac intensive care in the state of Pernambuco.

The surgical procedures were performed consecutively by four different surgical teams working in the Hospital. The on-call team was composed of paediatric intensive care specialists and paediatric cardiologists. All children were submitted to the same post-operative protocol.

This is an observational, retrospective and cross-sectional study. The statistical analysis was performed using SPSS 8.0 statistical software package. Outcome, defined as in-hospital mortality or survival, was correlated with each one of the chosen variables individually and through the final index score using the chi-square test corrected by Fishers' exact test when indicated, odds ratio, and confidence intervals. The Mann-Whitney test was used to compare the results obtained using the categories of surgical risk developed by Jenkins et al. as opposed to the Aristotle score. Receiver operating characteristic curves were used to calculate the sensitivity and specificity of the proposed score. The level of significance used was 5%. Outcome was not correlated to individual surgical expertise, this data not being available for analysis.

Results

Demographics: the ages of the patients varied from 20 hours to 16 years, with a mean of 3.3 years, standard deviation of 3.97, and median of 1.5 years. Their weights varied from 950 grams to 68 kilograms, with a mean of 13.4 kilograms, standard deviation of 11.46 kilograms, and median of 9.5 kilograms. Only 36 babies weighed less than 2.8 kilograms, and mortality in this group was 28%. The distribution was uniform for gender, with 413 children, or 51.2%, being male. The average number of surgical admissions for each year over the period of study was 163.60. Of the patients, a total of 120 died, giving in-hospital mortality of 14.7%.

In Table 2, we have shown the distribution of patients according to the studied variables, and the associated relative risks for death. Higher rates were observed in the subgroups with scores of two, this reaching statistical significance for all five variables studied.

Table 2. Distribution of variables according to those studied and their associated relative risks for death.

For age, the group with a score of zero was composed of 477 children older than one year. Their rate of mortality was 11.11%. The group scoring one included 220 children with ages between one and twelve months. Mortality for this group was 15.91%, with odds ratio of 1.51 and 95% confidence intervals from 0.93 to 2.46 (p equal to 0.0989). The group with a score of two included 121 neonates. This group suffered 26.45% mortality, with an odds ratio of 2.88, and 95% confidence intervals from 1.70 to 4.85 (p less than 0.0001). The most common surgical procedures performed in the neonatal period were systemic to pulmonary shunts in 34 babies, arterial switch operations in 29 babies, and repairs of aortic coarctation in 20 babies.

Almost three-fifths of our population was well nourished, specifically 58%, and mortality was 10.62% in this group. A quarter, or 206, of the children underwent surgery with weights below the 5th centile for age and sex. This group suffered 24.27% mortality, which corresponds to an increased risk of 2.70, 95% confidence intervals ranging from 1.71 to 4.26 (p less than 0.0001). There was also an increased rate of mortality, of 14.18%, in those undergoing surgery with lesser degrees of under-nourishment, but this did not reach statistical significance (p equal to 0.3090).

When assessing the association with clinical risk factors, our group at low risk included 538 patients, accounting for two-thirds of our sample. Rate of mortality in this group, in the absence of associated clinical risk factors, was 5.95%. In the intermediate group, with one associated risk factor, we placed 204 children, and the mortality was 23.04%, with an odds ratio of 4.73, and 95% confidence intervals from 2.84 to 7.90 (p less than 0.0001). The highest mortality for all variables analyzed in this study, at 53.95%, was seen in the 76 patients making up the group with more than one associated clinical risk factor. Odds ratio for this value was 18.52, with 95% confidence intervals from 10.03 to 34.36 (p less than 0.0001).

As shown in Table 3, the eight clinical risk factors included in the index were also associated independently with mortality. An increase in mortality was observed in the presence of each of the risk factors, and this reached statistical significance for all but an associated infection. The most common risk factors before surgery were severe clinical cyanosis, present in 121 patients, with mortality rate of 28.10%, odds ratio of 6.18, and 95% confidence intervals from 3.51 to 10.90, and pulmonary hypertension, present in 65 patients, with a mortality rate of 38.46%, odds ratio of 9.88, and 95% confidence intervals from 5.12 to 19.11. The presence of genetic syndromes and severe cardiac failure was also associated with mortality, with odds ratio of 10.10 and 10.01, and confidence intervals from 5.12 to 19.96 and from 4.83 to 20.78, respectively. Pre-operative hospitalization for more than one week showed a mortality rate of 45.16%. Mechanical ventilation, as well as acidosis, was also associated with very high rates of mortality, but the numbers of patients with these risk factors was very small.

Table 3. Associated clinical risk factors and mortality after paediatric cardiac surgery.

In Table 3, we show the rates of mortality according to the level of surgical complexity, and calculated using both the classification proposed by Jenkins and colleagues and the Aristotle score. Almost two-thirds of all patients scored zero using either classification. We included 548 patients in this group, with 8.76% mortality, when the procedures were stratified according to the risk categories established by Jenkins et al., and 550 patients with 9.45% mortality when they were stratified by the Aristotle score. When using the criterions proposed by Jenkins et al., we had only two patients scoring two, and hence considered at high risk. Both were neonates submitted to a first stage of the Norwood procedure, and both died. The remaining 268 children were stratified within the third or fourth categories established by Jenkins et al., and hence attributed a score of one in our study. They suffered 26.12% mortality, with odds ratio of 3.68, and 95% confidence intervals from 2.42 to 5.62 (p less than 0.0001). When using the Aristotle score, we placed 206 patients in the category allocated a score of one, and 62 patients in the category allocated a score of two. Their rates of mortality, respectively, were 22.33%, with odds ratio of 2.75 and 95% confidence intervals from 1.74 to 4.35 (p less than 0.0001), and 35.48%, with odds ratio of 5.27 and 95% confidence intervals from 2.79 to 9.92 (p less than 0.0001).

Cardiopulmonary bypass was used in 591 patients, or 72.25% of our sample, with mean time of 52.79 minutes, and standard deviation of 43.73. Mortality was 5.73% in those not requiring cardiopulmonary bypass, as opposed to 12.18%, with odds ratio of 2.28, and 95% confidence intervals from 1.17 to 4.52, when bypass was used for up to 90 minutes, and 33.54% mortality, with odds ratio of 8.31, and 95% confidence limits from 4.19 to 16.75, when its use exceeded 90 minutes.

Finally, the overall index was calculated twice, once using the categorization of surgical complexity suggested by Jenkins et al., and again using the Aristotle surgical score. There was a linear increase in mortality with the increase of the final score, regardless of which criterion for surgical risk was chosen. Application of the Mann-Whitney test showed no statistical difference between the results when the index was calculated using either approach (Table 1). No patients died when the score was zero, but mortality increased from around 10% with a score of three, to close to 30% with scores of five and six, and to over 50% with a score of eight. None of our patients reached scores of 10, and more than three-fifths of all patients had scores between zero and three, independent of the way in which the index was calculated.

Receiver operating characteristic curves were generated to assess the sensitivity and specificity of the index, and to define groups specifically at risk. The sensitivity of the index is maximized when two categories are stratified with the cut-off point at a score of two, with 95% confidence intervals from 82.59 to 94.08. In this situation, the sample is stratified in two nearly equal halves, with 376 patients, or 46%, deemed to be at low risk, with a rate of mortality of 3.72%, and 442 patients, or 54%, falling in a group at high risk, with a rate of mortality of 23.98% (Table 4). The maximal specificity of the index in predicting adverse outcome, however, was achieved when the cut-off between the categories deemed at low or high risk was set at a score of six, with 95% confidence intervals form 93.53 to 96.73. The group deemed at high risk in this fashion suffered 48.48% mortality, but the group contained only 66 children, or 8% of the overall population, corresponding to an 85.1% reduction in the group considered to be at high risk (Table 5).

Table 4. Relative risks of death maximizing the sensitivity of the Index.

Table 5. Relative risks of death maximizing the specificity of the Index.

Discussion

One of the major difficulties in assessing the risk of death subsequent to paediatric cardiac surgery is the great variety of surgical procedures, with many different levels of technical complexity involved in the repair of the multiplicity of congenital cardiac malformations. Added to that, the act of surgery in itself is unlikely to be the sole determinant of outcome in any environment. In developing countries, this is probably more so, as there are many other challenges to face. In our experience, serving as a tertiary regional centre in Northeastern Brazil, we perform surgery on a subset of patients often with serious associated conditions, such as advanced malnutrition, severe cyanosis, or pulmonary hypertension due to late diagnosis or referral.

The objective of our study was to determine, using a set of simple clinical and surgical criterions, if an adverse outcome, defined as death in hospital of children undergoing cardiac surgery, could be predicted at the time of admission to the intensive care unit. Our reason for performing the study was the need to establish a simple measure of outcome that could be of value in similar units in other developing countries, where much sophisticated data is not available on a routine basis, due to limitation of economy or technology. We found that all our chosen variables demonstrated an independent association with outcome, with a worse prognosis found in those placed in the categories with higher scores.

Those with the worst prognosis were children undergoing surgery in the neonatal period, below the 5th percentile for weight, and submitted to more complex surgical procedures or to periods of cardiopulmonary bypass in excess of 90 minutes. Although the increase in mortality for children operated with less good nourishment did not prove to be statistically significant, it is our opinion that nutritional state remains a very important issue when undertaking cardiac surgery in children. Since our numbers were relatively small, and from one single centre, we have opted to maintain the three nutritional categories as originally planned for further analysis.

In our original design for our index, we intended to use the four categories for surgical risk suggested by Jenkins et al. in 1995.¹¹ Later, with the extension to six categories of risk,³⁰ and with the appearance of the Aristotle score,²⁸ we re-calculated the data for all patients twice, using both of these new criterions, and compared the final results. Although we found no statistical difference using either of the surgical scores, the Aristotelian approach produced a more even distribution of our patients into categories of low, intermediate, and high risk. This may be because Jenkins et al. created their highest categories to include only a small number of very complex surgical procedures. Indeed, we had no patients in their fifth category, and only two in the sixth category. Both were neonates in whom the first stage of the Norwood sequence of procedures was attempted unsuccessfully. It is possible that the calculation of this index in units dealing mostly with procedures of lesser technical complexity, such as our own, will lead to similar results. Because of this, we have opted to use the Aristotle score as our variable for surgical complexity when analysing the sensitivity and specificity of our index.

Although the period of cardiopulmonary bypass may be seen as a surrogate marker for surgical complexity, we have observed that, in the presence of an unexpected finding in the operating room, or in the hands of a less experienced cardiac surgeon, the time needed for cardiopulmonary bypass may be longer than expected for the lesion being corrected. This was the reason for including this variable in the index. The cut-off point of 90 minutes to separate the categories of average and high risk was determined subsequent to review of our data, which shows a two-fold increase in mortality when bypass is used beyond this time.

There are other variables, such as prematurity, or the use of prostaglandins in the pre-operative period, which could have been added to the index. In our practice, they would account for a very small number of patients, this being the reason why we did not choose them in the first place. The lowest body temperature achieved during surgery, or the time of aortic cross-clamping, are also variables that could have been chosen to compose the index, but their inclusion could also amplify the surgical risks three to four folds. We opted not to use them.

The stratification of the variables into categories of low, intermediate, and high risk could also have been expanded in an attempt to increase the sensitivity and specificity of the index. When deciding on its composition, however, we bore in mind that the more complex an index, the longer the time and attention required to collect data precisely, and the less likely its chance to be incorporated in clinical practice in a busy unit. Hence, we chose just three basic categories.

The presence of pre-operative clinical risk factors also influenced the outcome of the patients. All variables except associated infections correlated significantly with the deaths of our patients. Only 18 of our children underwent surgery during the course of an associated infection. This number is too small to be certain of its value, and because of this, we have retained it in the index.

Regarding pulmonary hypertension, the reason for using the pulmonary arterial systolic peak pressure assessed by Doppler echocardiography rather than the pulmonary vascular resistance was the fact that cardiac catheterization is not performed routinely in these children in this part of the world.

A pre-operative period of hospitalization of more than a week was chosen when creating the index because, in this part of the world, it is not uncommon for the poor, sick, children to be admitted to general paediatric hospitals due to a deterioration in their clinical condition, and for them to stay in these hospitals for weeks or months due to the limited number of beds in paediatric cardiac units. They often develop complications, such as anaemia or recurrent infections, and require various manipulations, like blood transfusions and the use of different antibiotic schemes. Not surprisingly, high rates of death were observed in this group of patients.

The volume of patients submitted to surgery has previously been shown to influence outcomes. Our Centre qualifies as a unit with high volume,¹ performing as it does more than 100 surgical procedures each year. In this sample, we have encountered some of the typical features of units coping with high volumes of patients, such as an elevated percentage of procedures performed in the first year of life (44%), of associated clinical risk factors (44%) and of operations of moderate to high levels of complexity (33%).

The number of procedures undertaken by the individual surgeon has also been shown to influence outcome.¹ This data was not available for analysis in our study, but it has been noted that four different surgical teams performed these operations, and that the total number of cases per surgeon, for the most part, fell below the suggested number of 50, a factor which may have influenced our results.

The total mortality for our patients was high at 14.7%. To some extent, this may reflect the learning curve of the surgical and post-operative teams, as the period of study commenced concomitant with the establishment of our unit. It may also be related to the clinical condition of some of the patients, as mortality for patients without associated clinical risk factors was 5.95%, regardless of age or surgical complexity.

Overall, the index we have created was closely associated with in-hospital mortality. Using receiver operating characteristic curves, we were able further to stratify groups deemed to be at low, intermediate, or high risk of death. The groups at low risk included patients scoring from zero to two. Their rate of death was below 4%, and 376 children, or almost half of our sample, were in this group. The group at high risk included those children scoring more than six. The rate of mortality for the 66 children in this group was 46%. In the future, when faced with such high scores having calculated our index, our team may be prompted to re-assess ways of improving the clinical condition of the child, and reduce intra-operative risks, prior to deciding on the strategy for management. The group at intermediate risk also accounted for almost half of our sample, containing the 376 children who scored from three to six. This group may be a harder one to advise and manage in the future, although we feel that, on the basis of our initial results, it is fair to evaluate the potential benefits of changing the final score achieved in the index prior to submitting any individual child to cardiac surgery.

Predicting a bad outcome in an ever changing environment is certainly a major challenge, but it must be faced as a means to improve the quality of our work. In our study, we assessed adverse outcome in terms of death in hospital, this continuing to be the reality we face, working in a part of the world where paediatric cardiac intensive care units are not always seen as a priority, and where the pre, intra, and post-operative management of these children is, for the most part, not standardized.

When, as we hope, mortality reduces and begins to get closer to zero, the definition of adverse outcome may be shifted towards morbidity, such as prolonged stay in the intensive care unit. Even in this scenario, the index we have created could still, potentially, remain predictive of adverse outcomes.

Whereas complex and comprehensive surgical scores, such as the Aristotle score,²⁸ may be helpful to shed light into the overall prognosis of children undergoing cardiac surgery, they focus primarily on surgical complexity. The index that we have created considers other aspects, such as nutritional state and associated clinical risk factors. We hope it may prove helpful not only for ourselves, but also for those working in other units in developing countries, where the conditions faced by ourselves are more prevalent. These ever changing situations, and different stages of expertise in paediatric cardiac surgery, point perhaps to the need to form an international working group, made up of clinicians and cardiac surgeons, to examine different surgical realities, and to generate a more universally applicable index, which could give us a better picture of mortality and morbidity for children undergoing cardiac surgery throughout the world.

Our current index was developed in a single centre, with a relatively small number of patients, and with four different surgical teams at work. At this stage, it may be necessary to carry out an analysis of our results in greater depth, as well as correlating our data with other well established indexes for general intensive care. Furthermore, studies from other centres, as well as multi-centric studies, would be valuable in establishing the significance of our findings, and further determining whether getting our index will, or will not, become valuable elsewhere in assessing the likelihood of adverse outcomes from cardiac surgery undertaken in children.

Acknowledgements

We thank Dr Vera Aiello, form the Heart Institute of São Paulo, Brazil, for revising the manuscript, as well as giving valuable suggestions during its preparation.