Early recognition of clinical deterioration in paediatric patients regularly challenges health care professionals. Therefore, paediatric early warning systems as part of an overall clinical response system are recommended by the European resuscitation council Reference Perkins, Gräsner and Semeraro1 as well as by the International Liaison Committee on Resuscitation. 2 Paediatric early warning scores combine the observation of vital signs and clinical indicators by a defined scoring system that is supposed to lead to an escalation algorithm regarding medical interventions. High total scores are considered a cause for concern and may indicate a patient is at risk for clinical decompensation. Reference Monaghan3–Reference Chapman, Wray, Oulton, Pagel, Ray and Peters5 It is designed to improve communication and to help medical staff to recognise deteriorating patients at an early stage. Reference Lambert, Matthews, MacDonell and Fitzsimons6,Reference Chapman and Maconochie7
Several different paediatric early warning scores have been developed in the past. Reference Chapman, Wray, Oulton, Pagel, Ray and Peters5,Reference Chapman and Maconochie7 Some countries in Europe like the United Kingdom, the Netherlands and Ireland recommend the implementation of paediatric early warning scores on a national level. Reference Chapman, Wray, Oulton, Pagel, Ray and Peters5,Reference Alten, Klugman and Raymond8,Reference Olsen and Puri9
Children with cardiac disease are an especially vulnerable group; in-hospital cardiac arrest occurs in 2.6–6% and is associated with significant morbidity and mortality. Reference Alten, Klugman and Raymond8 However, most of the paediatric early warning scores cannot be applied in routine clinical practice in children with cyanotic heart disease due to their different baseline oxygen saturation. Reference Olsen and Puri9 Until now, only one score system especially designed for paediatric cardiac patients has been described. Reference McLellan and Connor10 This score defines individual baselines with relative deviations for each patient. However, to avoid calculation errors, this score probably works best with an electronic recording system for implementation into daily routine, which is not yet available everywhere. Therefore, the aim of the present study was the development and evaluation of a modified paediatric early warning score, which does not require electronic calculations.
Methods
Besides the modified paediatric early warning score, additional data were recorded: patient age, diagnosis, weight, reason for admission and any performed interventions because of clinical deterioration. All patients admitted to the paediatric cardiology ward during the study period were included. The only inclusion criterion was the presence of cardiac disease. Data of all modified scores that were obtained during the 4 months observation period were collected and prepared for analysis.
In April 2020, the paediatric cardiology ward of the University Hospital of Cologne introduced the modified paediatric early warning score. This scoring system was based on the original Irish paediatric early warning score, 11 which consists of eight bedside collected clinical parameters of different organ systems (“Respiratory,” “Cardiovascular,” “Behaviour/Neurology”) and an additional parameter: “nurses or parents concern” (Table 1). Due to different normal values, each age category (0–3 months, 4–11 months, 1–4 years, 5–11 years and 12+ years) had a separate form. The measured values were rated on a scale from 0 to 3 (Table 1 shows an example of a paediatric early warning score form for the age group of 0–3 months). “Supplemental O2” was calculated by multiplication of inspired oxygen fraction (%) and flow (l/min). The modification consisted in adjusting the “normal” oxygen saturation levels for children with cyanotic heart defects, as their baseline oxygen saturation is lower compared to children without cyanotic cardiac defects. The children were categorised at the time of admission according to their cardiac defect in:
Parameters and score points for the calculation of paediatric early warning score (example for the age of 0–3 months)
AVPU=alert, voice, pain, unresponsive; BP=systolic blood pressure; CRT=capillary refill time, Syst.; HR=heart rate; RR=respiration rate; Supplemental O2=supplemental oxygen therapy; SpO2=oxygen saturation; WOB=work of breathing.
-
“non-cyanotic” (normal oxygen saturation > 95%),
-
“mild cyanotic” (normal oxygen saturation: 85–95%), e.g., Fallot
-
“severe cyanotic” (normal oxygen saturation: 75–85%), e.g., hypoplastic left heart syndrome.
According to the total calculated score (range of 0–22), actions are prescribed by an algorithm. Higher scores indicate patients with an increased risk of clinical deterioration. The score recommendations for time intervals of re-evaluation and action were slightly adapted due to reduced nurse-patient relation compared to the original Irish score:
-
Basic evaluation once per shift, which corresponded to every 8 hours
-
Total scores values of 2–3: re-evaluation every 4 hours
-
Total scores value of 4–6: extra assessment of at least a doctor on call and a re-evaluation every 1–2 hours
-
Total score values ≥ 7 as a predictor for an acute deterioration: “Urgent call” with notification of the local response team.
It is important to note that not only high score values were supposed to be a reason to contact the responsible clinicians.
The resulting medical interventions that were performed included:
Airway/breathing interventions
-
Increase of nasal flow rate of air-oxygen-mixture
-
Increase of inspired oxygen fraction
-
Nebulisation of beta-agonistic drugs
-
Change of positioning (e.g., upper body elevation)
Circulatory interventions
-
Fluid challenge
-
Transfusion of blood products
-
Change of hemodynamically acting drugs
Other interventions
-
Interventions according to sepsis algorithm (blood cultures, antibiotics)
-
Administration of drugs (e.g., pain medication, sedatives)
-
Involvement of paediatric intensive care team
-
Individual adjustment of acceptable critical score values or individual adjustment of assessment intervals (different from the general rules), if the patient is in acceptable conditions despite elevated score values (e.g. if critical high score values were mainly caused by low, but acceptable oxygen saturation values).
The study was performed to validate the modification for adequate detection of critically ill patients. Therefore, the distribution of score values between the original uncorrected paediatric early warning score and the modified score system was compared and cut off points for critical deterioration or need for transfer to the ICU were analysed.
Statistical analysis was performed using SPSS statistical package, (release 28.0; IBM, Armonk, NY, USA). A consultant of the Institute of Medical Statistics and Computational Biology, Faculty of Medicine, University of Cologne reviewed it.
Descriptive data were given as median and interquartile range or total range.
For comparison of the non-normally distributed score values, Mann-Whitney-U- test was performed. To measure accuracy and ability of the paediatric early warning score to screen patients in need of an intervention or transfer to the ICU, receiver operating characteristic curves were generated. For evaluation of differences between receiver operating characteristic curves, confidence intervals were calculated. Sensitivity, specificity and positive predictive values were calculated. Sensitivity is defined as the probability that the early warning score detects patients needing medical intervention or unplanned transfer to the ICU. Specificity for this study is defined as the probability that modified score below a certain level excludes the need for medical intervention or unplanned transfer. Positive predictive value is defined as the probability that a patient will be in need of medical intervention or unplanned transfer to ICU, reaching a certain score value.
Results
During the study period, 106 patients with 118 hospital stays were included. Twelve patients had multiple admissions, two patients even with different indications for admission during the study period; each of these admissions was recorded separately.
In total, 2180 records of paediatric early warning scores could be analysed. Unplanned admission to the ICU was documented three times. Due to palliative care and thus alarming score values without following intervention, one patient was excluded from the analysis of sensitivity and specificity.
Demographic data
Table 2 shows the demographic data of the patients. Three patients were deceased before discharge. No cardiopulmonary arrests occurred in the ward during the study period.
Demographic data
Hypoplastic left heart syndrome or other types of single ventricle physiology were the most common primary diagnosis in our cohort patients, severe cyanosis was present in 22% of the patients.
Distribution of scores
In 50/118 (42%) admissions, the scores were always < 4 during the entire stay; in 101/118 (86%), all the scores were < 8. In patients without cyanotic heart disease, 17/67 (25%) of the recorded values were ≤ 1 and 25/67 (37%) ≤ 2 compared to 3/51 (6%) and 5/51 (10%) in patients with cyanotic heart disease. Most interesting for identification of patients at risk are score values ≥ 4. Using the original score, the distribution of the frequency of recorded values showed a significant shift towards higher score values for patients with cyanotic heart disease (Mann-Whitney-U Test, p < 0.001) (Fig. 1). This shift is also represented by the median score value, which is two (interquartile range 0.8–3.0) for non-cyanotic and three (interquartile range 2.0–4.0) for cyanotic patients. Applying the modified score, a median of two (interquartile range 1.0–3.0) is equal for both groups and the distribution of score values approaches that for patients without cyanosis. Excluding the lowest two score values (0 and 1), which are clinically not relevant, there was no statistical difference between cyanotic and non-cyanotic patients (Mann-Whitney-U Test, p = 0.581).
Distribution of paediatric early warning score values with modified and non-modified score (n = 2180).
Evaluation of cut off points, sensitivity and specificity
For the analysis of sensitivity and specificity, receiver operating curves were created (Fig. 2); one patient with palliative care and consistently high paediatric early warning scores had to be excluded for this analysis. The area under the receiver operating characteristic for patients without cyanotic heart defects modified score is 0.908 with a 95% CI of 0.862–0.954 for the original as well as for the modified score.
Receiver operating characteristic curves regarding in terms of an intervention.
For patients with a cyanotic cardiac defect, using the original score, the area under the receiver operating characteristic is 0.730 (95%-CI 0.636–0.824), compared to an area under the receiver operating characteristic for the modified score of 0.862 (95%-CI 0.809–0.915). While there is a significant difference in the area under the receiver operating characteristic for patients with and without cyanotic heart defect when using the original score (p = 0.001), there is no significant difference when using the modified (p = 0.207).
The calculation of the Youden-index as the optimal combination of sensitivity and specificity resulted in a threshold value for the modified score of four. Analysing sensitivity and specificity combined for all patients (cyanotic and non-cyanotic) and for this threshold of four, the modified score has a sensitivity and a specificity both of 81.4%, whereas using the original score, sensitivity was only 78.6% and specificity 70.7%. In patients with cyanotic cardiac defects sensitivity and specificity of the original score were 72.7% and 58.4%, compared to 78.8% and 78.2% using modified.
In 72 of 2108 (3.42%) paediatric early warning score recordings, an intervention was necessary. A detailed analysis of the scores at which an intervention took place showed that 44/72 (61.1%) of the interventions took place at scores in the range of 4–7, 15/72 (20.8%) at scores ≥ 8, but 13/72 (18.1%) also at scores < 4. In 50% of the cases that scored ≥ 4, a medical intervention was not necessary and an individually acceptable score value was defined. Total score values ≥ 8 were present 29 times; all patients that were transferred to the ICU had a score ≥ 8. The area under the receiver operating characteristic for the modified score of the three patients with an unplanned transfer to the ICU was 0.997.
Analysis of the additional parameter “drug withdrawal”
Drug withdrawal had a significant influence (p < 0.001) on the score values. There was a shift towards higher score values in children with withdrawal symptoms. The median score increased in patients with withdrawal from two to three; (p < 0.001). The proportion of score values > 4 was 10.5% in patients without withdrawal versus 16.4% in patients with withdrawal.
Discussion
To our knowledge, the present study is the first study to examine the ability of a paediatric early warning score modified for children with cyanotic CHD without the need of electronic aids.
Early warning scores have been recommended at a national level in several countries. They are recommended by the resuscitation guidelines of the European Resuscitation Council and by the “Consensus on Science with Treatment Recommendations” of the International Liaison Committee on Resuscitation. 2 Although paediatric early warning scores have not yet been shown to significantly reduce mortality and morbidity, there is agreement that they are one component in a system of interventions designed to improve patient safety. Reference Roland, Powell and Lloyd12 One of the advantages is a standardised language between nurses and doctors. Reference Andrews and Waterman13 In the United Kingdom, paediatric early warning scores are established in many hospitals and there is a national initiative to develop a uniform paediatric early warning score 14 . In the Netherlands, a recommendation by the Ministry of Health to introduce paediatric early warning scores was issued in 2011. Reference de Groot, Damen and de Loos15 Only in Ireland a uniform paediatric early warning score is present, developed by the Royal College of Physicians with extensive material for implementation. 11 Therefore, we chose this scoring system as a basis. However, like almost all existing score systems, it does not consider the deviating normal oxygen saturation values of children with cyanotic cardiac defects. Using the original score would lead to many false-positive high scores and trigger unnecessary reassessments or interventions. Not only would the increased workload of nurses be a problem but also the dissatisfaction with such a scoring system, making its implementation more difficult.
The present study attempts to adapt the well-designed Irish paediatric early warning score for patients with cyanotic heart disease without the use of electronic aids as it is applied by the Cardiac Children's Hospital Early Warning Score, Reference McLellan and Connor10 the only early warning score suitable for paediatric patients with cardiac disease.
In contrast to most other studies, we did not choose the criterion “cardiac arrest,” “respiratory decompensation” or “unplanned ICU-transfer” for evaluation of effectiveness, Reference Chapman and Maconochie7,Reference McLellan and Connor10,Reference Parshuram, Duncan and Joffe16 as we had no cardiac arrest during the time of evaluation and only very few unplanned emergency transfers to the ICU. Therefore, the criterion “need for medical intervention” was used as endpoint in the present study.
In the first step of analysis in patients without cyanotic cardiac disease, the area under the receiver operating characteristic showed that a score value ≥ 4 had the best combination of sensitivity and specificity, using the Youden-index for analysis. Therefore we could use this score value as threshold for the need of medical intervention and it was exactly the same threshold given in the user manual of the Irish score as “Urgent medical review.” 17
With the modified paediatric early warning score, we were able to adapt the original scoring system in a way that the same threshold could be achieved for patients with cyanotic cardiac disease. For patients with CHD, the modified paediatric early warning score was able to identify patients at risk for clinical deterioration and need for medical intervention with better sensitivity (0.79 versus 0.73) and especially better specificity (0.78 versus 0.58) than the original scoring system.
However, the presence of a critical score value alone did not necessarily lead to an action. After notifying clinician in charge in case of a critical value, the decision to intervene was always made after reassessing and examining the patient by the doctor on call. Sometimes children with higher score values were considered stable and therefore the assessment interval was adapted to the patient’s requirement and did not correspond to the paediatric early warning score algorithm itself.
The analysis of score values, at which an intervention was performed, showed that early warning scores are only an additional tool, as in some cases, interventions were already carried out at scores lower than the threshold score and, conversely, that even at higher scores an intervention was not always necessary. In this respect, paediatric early warning scores are only an additional instrument and cannot replace the individual assessment of the patient.
Regarding the acceptance of the paediatric early warning score, it can be stated that although it was accompanied by some more paperwork, the nursing staff mostly welcomed it, as there was a clear algorithm for how to respond to critical score values.
The two major limitations of the study are the retrospective design and the data basis of a single centre. Nevertheless, the modified paediatric early warning score should be representative, as the data were collected at a university hospital with all kinds of paediatric cardiology patients as well as paediatric cardiac surgery (except heart transplant).
The small number of patients who required an unplanned transfer to the ICU did not allow a statistical analysis of the criterion “unplanned ICU transfer” and inhibited a comparison with studies using this endpoint.
A general limitation of paediatric early warning scores is that, in addition to a slowly developing deterioration of patients, sometimes sudden, acute problems can occur without warning signs (e.g. a pneumothorax). Therefore, it is important to emphasise that the score is only an additional tool to identify patients at risk, but that there are also situations that require the immediate activation of a rescue team without prior scoring. In the present study, two of the few unplanned ICU transfers were of this type.
We showed that the parameter “drug withdrawal” has a statistically significant influence on the total score value by influencing most parameters measured by the paediatric early warning score. Clinical deterioration, caused by the withdrawal symptoms must be taken seriously and the drug withdrawal must be treated urgently as it can easily lead to the patient’s decompensation (e.g. congestive heart failure or pulmonal hypertension is present). Then the score requires reevaluation within a short time interval.
Conclusion
The modification of the original Irish paediatric early warning score for patients with cyanotic cardiac disease provides a tool which can be used consistently for all patients on a paediatric cardiology ward. It enables the identification of patients in need of medical intervention with a high sensitivity and specificity and is one component in a complex system to improve patient safety.
Acknowledgements
We would like to thank the nurses of the paediatric cardiology ward of the University Hospital of Cologne for the intensified using and documenting the modified paediatric early warning score during the study period.
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Competing interests
None.
Ethical standards
This retrospective study was approved by the responsible ethics committee of the University Hospital of Cologne, Cologne, Germany (20-1485, 20th Dec 2020, Chairperson Prof. Dr R. Voltz). The medical records of all children who were admitted to the paediatric cardiology intermediate care unit between May 1, 2020 and August 31, 2020 were reviewed. The ethics committee waived the requirement to obtain informed consent from parents to review the data. The study was conducted in accordance with the World Medical Association Declaration of Helsinki. 18
