Pulse oximetry is the standard non-invasive technique used to measure oxyhaemoglobin saturation (SpO2). Its use is recommended as a basic standard of care by the Association of Anaesthetists of Great Britain & Ireland and the American Society of Anesthesiologists during anaesthesia and in critical care units. 1 , 2 These probes and associated software programmes are calibrated in healthy volunteers between arterial saturations of ~75 and 100%.Reference Kamat 3 Saturations below this are derived by extrapolation, which may affect the accuracy. The USA Food and Drug Administration (FDA) 510(k) standards state that a pulse oximeter should have an accuracy root mean square (Arms) of <3% when measuring SpO2 between 70 and 100%. 4 The Arms is a measure of overall accuracy and is calculated from the precision and bias. No standards exist for SpO2 below 70%. The gold standard method for the measurement of oxyhaemoglobin saturation is multi-wavelength co-oximetry (SaO2),Reference Jensen, Onyskiw and Prasad 5 which requires arterial blood samples.
Children with cyanotic congenital heart disease commonly have oxyhaemoglobin saturations below or at the lower end of the calibration range for peripheral pulse oximeters. Despite the lack of data regarding the accuracy of conventional pulse oximeters in this context, measured values of SpO2 are used to inform important clinical decisions. 2 , Reference Schmitt, Schuetz and Proeschel 6 They are also used to screen for cyanotic congenital heart disease in newborns.Reference Prudhoe, Abu-Harb and Richmond 7
Recent data have questioned the accuracy of pulse oximetry in animal models,Reference Dawson, Bastrenta and Cavigioli 8 , Reference Solevag, Dannevig and Saltyte-Benth 9 adults,Reference Dubowitz, Breyer and Lipnic 10 , Reference Milner and Mathews 11 and children.Reference Schmitt, Schuetz and Proeschel 6 , Reference Ross, Newth and Khemani 12 Ross et al found that accuracy varied significantly in a cohort of children with hypoxaemia of any cause as a function of the SpO2 range.Reference Ross, Newth and Khemani 12 Inaccuracy of pulse oximetry has significant implications, particularly in patients with cyanotic congenital heart disease whose baseline saturations are low.
We assessed the accuracy of the Masimo SET® LNCS Neo peripheral pulse oximeter in a cohort of children with cyanotic congenital heart disease.
We hypothesised that the peripheral pulse oximeter would be inaccurate at extremes of hypoxia.
Methods
Setting
The study was performed in a 14-bed Paediatric Intensive Care Unit within the University Hospital Southampton NHS Foundation Trust in the United Kingdom.
Subjects
We included children in the Paediatric Intensive Care Unit with an echocardiographic diagnosis of cyanotic congenital heart disease and an arterial line in situ. We excluded children with duct-dependent lesions to avoid the influence of a patent duct on the pre-ductal and post-ductal saturations.
Study design
Paired measurements of SpO2 (Masimo SET® LNCS peripheral pulse oximeter) and SaO2 (ABL800 FLEX analyser) were recorded prospectively whenever an arterial sample was taken for blood gas analysis. The decision to obtain the arterial sample was at the discretion of the nursing and medical team caring for the child. The arterial sample was analysed immediately upon obtaining the sample with no time delay. The SpO2 was recorded at the time the arterial sample was collected. The pulse oximeter had to have a stable recording for eight seconds after the removal of the arterial sample. This time period was specified to allow for the averaging time of the Masimo SET® LNCS peripheral pulse oximeter.Reference Ahmed, Rich and Finer 13 If the SpO2 changed during sampling, the recordings were discarded. The saturation probe position was routinely changed to prevent underlying skin injury to the child, as per our departmental protocol.
Statistical analysis
Data were analysed using MedCalc v12.7.7. Bland–Altman plots for multiple measurements per subject were used to determine the agreement between the Masimo SET® LNCS neo peripheral pulse oximeter (Massimo Corporation 2013, California, USA) and multi-wavelength co-oximetry.Reference Bland and Altman 14 The bias of the Masimo SET® LNCS pulse oximeter was calculated from the difference between SpO2 and SaO2. Mean bias reflects the average difference between SpO2 and SaO2 across the entire data set. The precision was calculated from the standard deviation of the mean bias. The accuracy root mean square (Arms) reflects both the bias and precision while correcting for the number of samples, and is used to describe the overall accuracy. Data were divided into two groups – above and below the lower calibration limit (SaO2 <75% and SaO2⩾75%Reference Kamat 3 ). Previous studies have all used SpO2 to make this division, which seems counter-intuitive as SaO2 is the gold standard.Reference Jensen, Onyskiw and Prasad 5 In our analysis, we have selected to compare the difference between the two methods against the SaO2 as the gold standard method, which is a recognised alternative approach to using the traditional Bland–Altman plots.Reference Krouwer 15
Results
Sample size
We obtained 527 paired measurements of SpO2 and SaO2 in 25 children, with a median age of 112 days (range 1 day–4 years). The lowest recorded SpO2 was 35%; the lowest recorded SaO2 was 28.9%. The data points on the Bland–Altman plots display the agreement between the two values of SpO2 and SaO2; each marker represents one pair of observations.
Overall accuracy of the Masimo SET® LNCS Neo pulse oximeter
The mean bias of the Masimo SET® LNCS Neo pulse oximeter was +4.7%±13.8% (1.96 SD). The large standard deviation indicates poor precision. The Arms was 3.3%. The Bland–Altman plot is displayed in Figure 1 and shows that the Masimo SET® LNCS Neo pulse oximeter both overestimates and underestimates oxyhaemoglobin saturations, but tends towards overestimation, sometimes by as much as 30%.
Bland–Altman plot for all data points (n=527).Reference Bland and Altman 14 Each symbol represents one patient with multiple measurements per patient.
Accuracy of the Masimo SET® LNCS Neo pulse oximeter when SaO2 was ⩾75%
In 270 paired samples, the SaO2 was ⩾75%, and the mean bias was +2.4±12.6%(1.96 standard deviation) (Fig 2). The Arms for SaO2 readings ⩾75% was 1.7%.
Bland–Altman plot for co-oximeter SaO2 recordings ⩾75% (n=270).
Accuracy of the Masimo SET® LNCS Neo pulse oximeter when SaO2 was <75%
In 257 paired samples, the SaO2 was <75% and the mean bias was +7.0%±13.7% (1.96 standard deviation) (Fig 3). The Arms for SaO2 readings <75% was 5.0%.
Bland–Altman plot for co-oximeter SaO2 recordings <75% (n=257).
Discussion
The accuracy of the Masimo SET® LNCS pulse oximeter diminished and the bias increased when SaO2 was ⩽75% in this study. Agreement with the co-oximeter was poor when SaO2 was <75%, with a tendency to overestimate oxyhaemoglobin saturation. This worrying observation is consistent with previously published data and leads us to question the use of these probes in children with cyanotic congenital heart disease.Reference Schmitt, Schuetz and Proeschel 6 , Reference Dawson, Bastrenta and Cavigioli 8 – Reference Ross, Newth and Khemani 12 Critical hypoxaemia may be undetected by these saturation probes and clinical decisions may be ill-informed. In our own unit, a critical incident occurred when clinicians were falsely reassured by SpO2 values.
To the best of our knowledge, only a limited number of studies have investigated the accuracy of pulse oximetry in children with critical congenital heart disease, all of which showed that accuracy is impaired as SaO2 falls.Reference Schmitt, Schuetz and Proeschel 6 , Reference Ross, Newth and Khemani 12 , Reference Torres, Skender and Wohrley 16 – Reference Das, Aggarwal and Aggarwal 19 SpO2 both overestimates and underestimates SaO2, with a tendency towards overestimation, which is consistent with the results of our study. Many studies have included children with non-cardiac causes of acute hypoxia, not the chronic hypoxia seen in children with critical congenital heart disease. A few studies have reported mean bias and precision, and only one other studyReference Ross, Newth and Khemani 12 has reported the Arms, which limits comparison between results. This study adds to the growing data highlighting concerns about the accuracy of pulse oximetry at the extremes of hypoxaemia, particularly in a cohort of children with critical congenital heart disease.
We acknowledge the limitations to our study. We included only a single probe for our data collection. The averaging time of peripheral pulse oximetry makes it difficult to guarantee simultaneous measurements of SpO2 and SaO2. We did not consider the influence of other factors that may have affected the measurement of SpO2, such as haemoglobin, pH, arrhythmias, and low cardiac output states. A much larger data set is required to enable multivariate analysis to consider these issues.
In conclusion, when assessing the degree of hypoxia in children with cyanotic congenital heart disease, strong preference must be given to direct co-oximetry on arterial blood samples when available. Peripheral pulse oximetry should be used with extreme caution and with awareness of its limitations. Better algorithms are required for these probes before unqualified recommendations can be made for their use in the care of children with cyanotic congenital heart disease.
Acknowledgements
We thank the nursing staff of the University Hospital Southampton Paediatric Intensive Care Unit for their assistance with data collection.
Financial Support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Conflicts of Interest
None.
Ethical Standards
Following a discussion with the Local Research Ethics Committee, the requirement for formal ethical approval was waived.
