Introduction
Pulse oximetry is a non-invasive tool that provides valuable information in various clinical settings.Reference Fouzas, Priftis and Anthracopoulos1 By measuring transmission of light through tissue and using the different absorption properties between oxygenated and de-oxygenated haemoglobin, pulse oximetry provides an estimation of arterial oxygen saturation in blood.Reference Fouzas, Priftis and Anthracopoulos1 Recent studies in adults and older children suggest that pulse oximetry may overestimate arterial oxygen saturation in Black patients compared to White patients, with hypoxaemia not being as frequently detected in Black patients.Reference Ruppel, Makeneni and Faerber2–Reference Burnett, Stannard and Wax10 In 2021, the Food and Drug Administration issued a Safety Communication describing the possible limitations of pulse oximetry and certain factors that may affect its accuracy, in particular skin colour.11
In the infant population, the risk of pulse oximeter inaccuracy in patients of various races and skin colours is less clear. One retrospective study in preterm infants found that pulse oximetry overestimated arterial oxygen saturation in Black compared to White infants, while another small prospective study in term infants with cyanotic CHD did not find skin pigmentation to affect pulse oximeter accuracy.Reference Vesoulis, Tims, Lodhi, Lalos and Whitehead7,Reference Foglia, Whyte and Chaudhary12
Newborn screening of critical CHD using pulse oximetry has become universal in the United States since its addition to the United States of America Recommended Uniform Screening Panel in 2011.Reference Oster and Kochilas13 It is typically performed in newborns around 24 hours following birth prior to discharge.Reference Martin, Ewer and Gaviglio14 Ineffective screening may lead to delayed diagnosis of significant CHD that requires timely intervention.Reference Oster and Kochilas13,Reference Martin, Ewer and Gaviglio14 The aim of our study is to evaluate whether there is a racial and ethnic discrepancy in the accuracy of pulse oximetry in the newborn period.
Materials and method
We performed a retrospective cross-sectional study at a quaternary hospital. We included neonates aged ≤ 7 days admitted to the Neonatal and Cardiac Intensive Care Units between January 2010 and June 2022 at the Children’s Healthcare of Atlanta. Baseline demographics, including parent-reported race and ethnicity, and clinical variables were extracted from the electronic medical record. This study was approved by our institution review board with a waiver of informed consent due to the retrospective nature of the study.
Eligible patients had an oxygen saturation level by pulse oximetry taken within 10 minutes of an arterial oxygen saturation measurement at or greater than 24 hours of age. Either Masimo LCNS Newborn or Neo-3 sensors were used for pulse oximetry. Arterial blood gas was obtained in the ICUs and analysed with a point-of-care blood analyser (i-STAT). Arterial oxygen saturation value was derived from measured partial pressure of oxygen, carbon dioxide, and pH. For each patient, the closest pulse oximetry saturation measurement was matched with the arterial oxygen saturation within the 10-minute period. Oxygen saturation bias (defined as pulse oximetry oxygen saturation—arterial oxygen saturation) and accuracy root mean square were assessed for each pair. A sensitivity analysis using a shorter 2-minute period was also performed. The frequency of occult hypoxaemia, defined as pulse oximetry saturation ≥ 95% with arterial oxygen saturation < 90%, was evaluated by race and ethnicity. The thresholds for occult hypoxaemia were determined based on the screening algorithm for critical CHD, in which a pulse oximetry saturation < 95% would prompt further investigation, either with a repeated pulse oximetry measurement or an echocardiogram.Reference Kemper, Mahle and Martin15 A secondary analysis was performed using a different set of thresholds of occult hypoxaemia adopted by several prior studies in children and adults,Reference Ruppel, Makeneni and Faerber2,Reference Andrist, Nuppnau, Barbaro, Valley and Sjoding6,Reference Henry, Hanson and Schulte8,Reference Sjoding, Dickson, Iwashyna, Gay and Valley9 defined as pulse oximetry saturation ≥ 92% with arterial oxygen saturation <88%.
Categorical variables were summarised using counts and percentages. Normally distributed continuous variables used means and standard deviations, while non-normally distributed variables used medians and interquartile ranges. Group comparisons for categorical variables used Pearson’s chi-squared test and Fisher’s exact test (when appropriate); continuous variables used the Kruskal–wallis rank sum test (groups > 2, non-normally distributed variables) and one-way ANOVA (groups > 2, normally distributed variables). Comparison with 2 groups used the Wilcoxon rank sum test and the students t-test for non-normally and normally distributed variables. Pairwise comparisons used two-way ANOVA with p-values adjusted for multiple comparisons using Tukey’s Honest Significance Test. Univariate logistic regression was used to find unadjusted prevalence ratios, and 95% confidence intervals were obtained for the relationship between occult hypoxaemia and race & ethnicity. The model was then adjusted for significant and clinically relevant covariates (age, gestational age, haemoglobin, CHD, respiratory support, and time difference). A Bland–Altman analysis was performed to assess agreement between pulse oximetry and arterial oxygen values. The above analysis was repeated for a subset of the sample who had a gestational age greater than or equal to 37 weeks and did not have respiratory support.
Mean bias and accuracy root mean square (below for formulas) were calculated to evaluate the accuracy of pulse oximetry. Bias was calculated by taking the difference between pulse oximetry and arterial oxygen saturations for each participant. Positive values indicate an overestimation. Accuracy root mean square was calculated by taking the square root of the squared bias. Mean bias and accuracy root mean square were then calculated for each racial group (Black non-Hispanic, White non-Hispanic, Hispanic, Other) and occult hypoxaemia group (yes/no) as well as for the subset analysis. To calculate the 95% confidence interval, we first calculated the standard error for the accuracy root mean square using 1000 bootstrapped resamplings for each group and/or subset and then manually calculated the 95% confidence interval.
For all statistical test, a p-value of < 0.05 was considered statistically significant. All analysis was performed in R Statistical Software (v4.2.1; R Core Team 2022).
$$ \rm Mean\ bias = {\sum_{\it{i}\,=\,1}^{\it{n}} \left( {\it SpO}_2 - {\it SaO}_2 \right) \over \it n} $$
$$\rm Accuracy\ root\ mean\ square = \sqrt{ {\sum_{\it {i}\,=\,1}^{\it n} \left( {\it SpO}_2 - {\it SaO}_2 \right)^2 \over \it n} } $$
$$95\%\, {\rm CI} = A_{{\it rms}} \pm 1.96 \times {\it SE}$$
Results
Cohort characteristics
There were 1645 infants included in our study, of which there were 639 non-Hispanic Black infants (38.8%), 651 non-Hispanic White infants (39.6%), 215 Hispanic infants (13.1%), and 140 other/mixed/unknown (8.5%). The mean age of the cohort was 2.6 days (standard deviation 1.5 days). The median gestational age for the cohort was 37 weeks (interquartile range 34.4–39 weeks). Hispanic infants had a greater median gestational age (37.6 weeks, 35.1–39 weeks) compared to non-Hispanic Black (37 weeks, 33–39 weeks) and White infants (37 weeks, 35–39 weeks). The majority of infants in the cohort had no CHD (71.4%), required respiratory support (81.1%), and had pulse oximetry oxygen saturation levels = > 90% (87.6%). The mean time difference between pulse oximetry and arterial oxygen saturation measurements was 0.8 minutes (2 min), with no significant difference between the groups. A summary of demographics and clinical variables is shown in Table 1.
Cohort characteristics by race & ethnicity
SpO2: pulse oximetry oxygen saturation; SaO2: arterial oxygen saturation.
Pulse oximetry accuracy
The mean bias for the cohort was 8.7% (standard deviation 10.5%). The mean bias was 8.8% (10.5%) for non-Hispanic Black infants, 8.6% (10.0%) for non-Hispanic White infants, and 8.7% (9.8%) for Hispanic infants (Figure 1). There was no significant difference between the groups (p = 0.91). Further subgroup analysis for oxygen saturation < 90% and = > 90% showed no significant difference in mean bias between the groups (p = 0.10 and 0.51, respectively). The accuracy root mean square over the entire range of the saturations was 13.6% (confidence interval 12.8–14.4%) for the cohort. The accuracy root mean square was 13.7% (12.6–14.8%) for non-Hispanic Black infants, 13.2% (11.9–14.4%) for non-Hispanic White infants, and 13.1% (10.8–15.4%) for Hispanic patients. Further subgroup analysis for infants who were term and in room air (176 infants) showed a mean bias of 7.5% (6.9%) and an accuracy root mean square of 10.1% (8.7–11.6%), with no significant difference between the groups (p = 0.71). In infants with pulse oximetry and arterial oxygen saturation measurements within 2 minutes or less (1439 infants), mean bias was 8.5% (10.4%), and the accuracy root mean square was 13.4% (12.5–14.3%), with no significant difference between the groups (p = 0.93). In the Bland–Altman analysis, an overall bias of 8.7% was confirmed, with a 95% confidence interval noted to be between −12.3–29.7% over the entire range of oxygen saturation (Figure 2).
Boxplot of mean bias between pulse oximetry oxygen saturation (SpO2) and arterial oxygen saturation (SaO2) by race and ethnicity.
Bland–Altman plot demonstrating pulse oximetry (SpO2) and arterial oxygen saturation (SaO2) agreement. Note the overall increase in saturation bias as SpO2 decreases.
Occult hypoxaemia
The frequency of occult hypoxaemia was 20.5% (337/1645 infants) in our cohort. The frequency of occult hypoxaemia was 22.4% (143/639) amongst Black infants, 19.3% (126/651) amongst White infants, and 19.0% (41/215) amongst Hispanic infants. Birth weight, gestational age, haemoglobin levels, and the presence of CHD were associated with occult hypoxaemia (Table 2). Unadjusted prevalence ratio for occult hypoxaemia was 1.16 (95% confidence interval 0.93–1.43, p = 0.18) in Black infants and 0.99 (0.72–1.35, p = 0.93) in Hispanic infants, compared to White infants. After adjusting for clinical variables (age, gestational age, haemoglobin, CHD, respiratory support, and time difference), the prevalence ratios for occult hypoxaemia in Black infants were 1.13 (0.92–1.40, p = 0.24) and 1.06 (0.77–1.45, p = 0.72) in Hispanic infants, compared to White infants. Excluding preterm infants (< 37 weeks), the adjusted prevalence ratio was 1.34 (0.97–1.85, p = 0.08) in Black infants and 1.12 (0.71–1.76, p = 0.63) in Hispanic infants, compared to White infants. A secondary analysis using an alternative definition of occult hypoxaemia (pulse oximetry oxygen saturation ≥ 92% with arterial oxygen saturation <88%) showed a 24.6% (405/1645 infants) frequency of occult hypoxaemia. The adjusted prevalence ratios in Black infants were 1.03 (0.90–1.18, p = 0.66) and 1.04 (0.86–1.26, p = 0.66) in Hispanic infants, compared to White infants. Excluding preterm infants (< 37 weeks), the adjusted prevalence ratio was 1.11 (0.83–1.50, p = 0.48) for Black infants and 1.20 (0.82–1.76, p = 0.34) for Hispanic infants, compared to White infants.
Cohort characteristics by occult hypoxaemia
SpO2: pulse oximetry oxygen saturation; SaO2: arterial oxygen saturation.
Discussion
In this study, the largest study to date in infants to evaluate the racial and ethnic discrepancy of pulse oximetry accuracy, we did not find a significant difference between non-Hispanic Black, White & Hispanic groups over the entire range of oxygen saturations, nor did we find a significant difference between Black and White infants in the frequency of occult hypoxaemia. Importantly, however, we did find that pulse oximetry substantially overestimated arterial oxygen saturation in our cohort, with a mean oxygen saturation bias of 8.7% and an accuracy root mean square of 13.6%. Similar to previous studies, we found pulse oximetry to be less accurate at lower oxygen saturation levels.Reference Vesoulis, Tims, Lodhi, Lalos and Whitehead7,Reference Aoki, Barrant, Gai, Handal, Xu and Mayrovitz16
The findings in our study differ from some of the adult and emerging paediatric studies,Reference Ruppel, Makeneni and Faerber2–Reference Burnett, Stannard and Wax10 which suggest that pulse oximetry may be less accurate in Black compared to White patients, possibly related to the variation in melanin composition, which contributes to the differences in skin pigmentation between races and ethnic groups. Melanin not only has significant light absorption at a wide spectrum of wavelengths, including the near-infrared spectrum used in pulse oximeter,Reference Matas, Sowa, Taylor and Mantsch17 but it can also exhibit substantial light absorption at the sensor site.18 Studies assessing skin pigmentation-related bias (as opposed to race and ethnicity) are limited, but increasing evidence suggests that pulse oximetry may overestimate oxygen saturation in patients with dark compared to light pigmentation.Reference Fawzy, Ali and Dziedzic19,Reference Gudelunas, Lipnick and Hendrickson20 In infants, however, the skin is not fully pigmented at birth because of the relatively low production of melanin in utero. Melanin continues to develop after birth and does not fully mature until about 6 months of age.Reference Walsh21,Reference Eichenfield, Frieden and Esterly22 The timeline for melanin maturation may explain the discrepant observations between adults, children, and newborns. Vesoulis et al. found a small but consistent increased overestimation of oxygen saturation by pulse oximetry in Black compared to White preterm infants;Reference Vesoulis, Tims, Lodhi, Lalos and Whitehead7 however, in premature infants born before 32 weeks gestation, there may not be much difference in skin pigmentation at birth between Black and White infants.Reference Post, Krauss, Waldman and Auld23 In the single study that evaluated skin pigmentation, Foglia et al. did not find skin pigmentation to significantly affect the accuracy of pulse oximetry in infants. This was a relatively small study of 35 infants, and notably infants with dark pigmentation were significantly older than those with light pigmentation (118 days versus 6 days).Reference Foglia, Whyte and Chaudhary12 Other factors such as hyperbilirubinemia can also affect skin pigmentation in infants. Bilirubin, however, has different light absorption properties from haemoglobin and is thought to have minimal effects on pulse oximetry.Reference Fouzas, Priftis and Anthracopoulos1,Reference Sinex24,Reference Chan, Chan and Chan25
In our cohort, the oxygen saturation bias (8.7%) and accuracy root mean square (13.6%) study exceed the Food and Drug Administration acceptable margin of error for pulse oximetry18 and are higher than described in previous studies,Reference Ruppel, Makeneni and Faerber2–Reference Vesoulis, Tims, Lodhi, Lalos and Whitehead7,Reference Foglia, Whyte and Chaudhary12 which range from 0.1 to 5.4%. One reason may be the use of a 10-minute window between pulse oximetry and arterial oxygen measurements. While some studies also used a 10-minute interval,Reference Savorgnan, Hassan, Borges and Acosta3,Reference Valbuena, Seelye and Sjoding5,Reference Andrist, Nuppnau, Barbaro, Valley and Sjoding6 several studies in children have either used simultaneous measurements or a much shorter time window.Reference Ruppel, Makeneni and Faerber2,Reference Vesoulis, Tims, Lodhi, Lalos and Whitehead7,Reference Foglia, Whyte and Chaudhary12 Our cohort included infants who were admitted to the ICU and likely critically ill, with potential for significant fluctuations in oxygen saturations within a short period of time. Sensitivity analysis using a 2-minute window demonstrated similar findings in our study. While the overall time difference between pulse oximetry and arterial blood gas was short (less than 60 s) and similar between groups, it may not be reflective of the pulse oximetry saturation at the time arterial oxygen saturation was measured. In addition, we used a single pair as opposed to an average of pulse oximetry saturations over a time window,Reference Savorgnan, Hassan, Borges and Acosta3 which may mitigate the effects of rapid fluctuations. The presence of fetal haemoglobin may also affect the bias between pulse oximetry and arterial oxygen saturations. Fetal haemoglobin is present in significant amounts at birth, and the genetic switch to adult haemoglobin happens over at least the first 6 months of life.Reference Pritišanac, Urlesberger, Schwaberger and Pichler26 While pulse oximetry is not affected by fetal haemogloin,Reference Fouzas, Priftis and Anthracopoulos1,Reference Chan, Chan and Chan25 arterial oxygen saturation measured by point-of-care blood analyser may be inaccurate in the presence of fetal haemoglobin. Point-of-care arterial oxygen saturation is a derived value from the partial pressure of oxygen using the oxygen-haemoglobin dissociation curve, which can be affected by unmeasured variables such as fetal haemoglobin.27 The oxygen-haemoglobin dissociation curve differs significantly between adult and fetal haemoglobin, which is shifted to the left in the presence of fetal haemoglobin.Reference Fouzas, Priftis and Anthracopoulos1,Reference Chan, Chan and Chan25 For the same partial pressure of oxygen, arterial oxygen saturation is higher for fetal haemoglobin compared to adult haemoglobin. Point-of-care blood analysers utilise the standard haemoglobin curve,27 which may significantly underestimate the arterial oxygen saturation.
There was also a high frequency of occult hypoxaemia in our cohort (20.5%), partly attributed to the large difference we found between pulse oximetry and arterial oxygen saturations. In previous studies in children, the frequency of occult hypoxaemia ranged from 4 to 21.1%.Reference Ruppel, Makeneni and Faerber2,Reference Savorgnan, Hassan, Borges and Acosta3,Reference Andrist, Nuppnau, Barbaro, Valley and Sjoding6 There is no uniform definition for occult hypoxaemia, making comparisons challenging. The thresholds are generally selected based on clinical relevance and the acceptable Food and Drug Administration margin of error.18 Most other studies defined occult hypoxaemia as pulse oximetry saturation ≥ 92% with arterial oxygen saturation < 88%.Reference Ruppel, Makeneni and Faerber2,Reference Andrist, Nuppnau, Barbaro, Valley and Sjoding6,Reference Henry, Hanson and Schulte8,Reference Sjoding, Dickson, Iwashyna, Gay and Valley9 Our secondary analysis using these oxygen saturation thresholds showed a slightly higher frequency of occult hypoxaemia (24.6%), with no significant difference between groups. The thresholds in our primary analysis were determined based on the cutoffs used in critical CHD screening.Reference Kemper, Mahle and Martin15 An infant with pulse oximetry saturation > 95% but arterial oxygen saturation < 90% would have a false-negative screening test, whereas an infant with pulse oximetry saturation between 90 and 95% would prompt further evaluation, with repeated pulse oximetry measurements and potentially an echocardiogram. The critical CHD screening algorithm also incorporates the comparison between pre-ductal and post-ductal measurements.Reference Martin, Ewer and Gaviglio14 For many patients in our study, the location of the pulse oximeter sensor was not documented in the electronic medical record system. The inclusion of this additional criterion into our definition for occult hypoxaemia potentially could affect the frequency of undetected hypoxaemia, though it would be unlikely to affect the evaluation for the presence of potential racial disparities. The high frequency of occult hypoxaemia in our study is notable, but its clinical significance is unclear and requires further investigation. In our univariate analysis, infants with CHD were associated with a higher incidence of occult hypoxaemia. Under-detection of hypoxaemia may potentially lead to delayed screening and diagnosis of CHD in the newborn period.
There were several limitations in our study. First, this is a retrospective study using parent-reported race and ethnicity data extracted from the electronic medical record system. While race and ethnicity can be a proxy for skin pigmentation, there can be heterogenous phenotypes with a spectrum of skin pigmentation. Furthermore, as mentioned previously, since infants are not born with their full pigmentation,Reference Walsh21,Reference Eichenfield, Frieden and Esterly22 race and ethnicity are less likely to be a reliable surrogate for skin pigmentation, particularly in this population. The understanding of the precise development of melanin and distribution remains limited, especially in infants and children,Reference Sharma, Brown and Powell28 which highlights the need for more objective and reliable assessment of skin pigmentation in future studies in this population. While there is currently no standardised approach to characterising skin pigmentation, there are some technologies that may allow for more objective quantification, such as colorimeters, diffuse reflectance spectroscopy, and optoacoustic imaging.Reference Sharma, Brown and Powell28 Photometric reflectance may be used to detect differences in skin pigmentation as early as 32 weeks postmenstrual age in premature infants.Reference Post, Krauss, Waldman and Auld23
Another potential limitation is that our infants, who were admitted to the ICU for various reasons, may have compromised perfusion. Pulse oximetry uses the pulsatility and variation of arterial blood volume during the cardiac cycle to detect oxygen saturation.11 Poor perfusion can result in intermittent drop-outs and may significantly affect the accuracy of pulse oximetry, especially in patients with dark pigmentation.Reference Gudelunas, Lipnick and Hendrickson20,Reference Giuliano, Bilkovski, Beard and Lamminmäki29 A significant number of infants in our study also required respiratory support, which may increase the overestimation by pulse oximetry.Reference Wackernagel, Blennow and Hellström30 In our sensitivity analysis, there remained no significant difference in oxygen saturation bias, accuracy root mean square or incidence of occult hypoxaemia between Black and White infants without any respiratory support.
As previously discussed, the location of the pulse oximetry sensor was not consistently charted. In the presence of a patent ductus arteriosus with right-to-left shunting, oxygen saturation may vary significantly between upper and lower extremities. The location of the pulse oximetry sensor, as well as from which the arterial blood sample was drawn, could potentially result in a differential between pulse oximetry and arterial oxygen saturation measurements. In the subset of infants with significant right-to-left shunting, oxygen saturations may be significantly higher in the arm compared to the leg, creating a potential for further overestimation of oxygen saturation by pulse oximetry if arterial oxygen saturation was measured from a post-ductal blood gas sample. Furthermore, the melanocyte content also varies significantly between different parts of the body. The fingernail bed, over which sometimes the sensor is placed, has roughly 5% of the melanocyte content of normal skin.Reference Güneş and Göktay31
Our study, the largest retrospective study in infants to date, did not find a systemic discrepancy in pulse oximetry between racial and ethnic groups as previously described in older children and adults. The evaluation of skin pigmentation, as an alternative to race and ethnicity, may more precisely detect melanin-related differences in pulse oximetry accuracy, especially in infants whose melanin continues to develop beyond birth. Additionally, we found that pulse oximetry significantly overestimated arterial oxygen saturation, and there was a high frequency of occult hypoxaemia overall. Future prospective studies that objectively measure skin pigmentation, assess the degree of distal perfusion and document the location of pulse oximetry measurements may be able to overcome some of the limitations of our study.
Acknowledgements
Thank you to Dr Rohali Keesari for her help with the preliminary analysis of the data.
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Competing interests
None.
