CHD is the most common form of birth defect, occurring in 6–8 of every 1000 U.S. live births.1,Reference Hoffman and Kaplan2 Critical CHD is a severe form of CHD requiring surgery or catheter-based intervention before age 1 year and occurs in 7200 newborns (1.8 of every 1000 live births) in the U.S. annually.1 Despite the seriousness of the disease, about 25% of newborns with critical CHD are discharged from hospitals after birth without a diagnosis.Reference Peterson, Dawson and Grosse3 In an effort to detect more cases of critical CHD before newborn discharge, newborn screening with pulse oximetry was added to the Recommended Uniform Screening Panel in late 2011.4 By 2018, all 50 states and the District of Columbia had implemented policies recommending or requiring screening.
A major challenge for implementing and sustaining mass screenings is false-positive test results, which lead to unnecessary consumption of healthcare resources, such as diagnostic echocardiograms. For other newborn screening tests, there have been findings of increased healthcare utilisation, including more frequent physician visits and hospitalisations among infants who received false positives compared to those who received true negatives.Reference Hayeems, Miller and Vermeulen5,Reference Karaceper, Chakraborty and Coyle6 The burden of false-positive screening results extends beyond unnecessary healthcare consumption, for example, long-lasting negative psychosocial effects of false-positive results on families have been reported in other newborn screenings.Reference Hewlett and Waisbren7–Reference Kerruish, Healey and Gray9
A very low false-positive rate in pulse oximetry screening has been reported.Reference Granelli, Wennergren and Sandberg10,Reference Plana, Zamora, Suresh, Fernandez-Pineda, Thangaratinam and Ewer11 However, these numbers were drawn from clinical research data, likely involving intensive training and supervision, or retrospective data from non-U.S. countries. False-positive rates are still unknown in real-world settings, specifically under mandatory state policies, where all hospitals, including those with low resources and less supervision, perform critical CHD screening.
This study evaluates changes in healthcare service utilisation following implementation of statewide policy mandates for pulse oximetry screening, specifically focusing on confirmatory echocardiograms ordered for newborns.
Materials and methods
Data sources
We used data from the 2008–2010 to 2012–2014 Maryland and New Jersey, and 2010 and 2014 Kentucky, Michigan, New York, and North Carolina, Statewide Inpatient Databases, Healthcare Cost and Utilisation Project, Agency for Healthcare Research and Quality.12 These states collectively account for approximately 18% of U.S. live births and were selected based on availability of data for public use, large populations, availability of necessary variables for analyses, and reasonable cost of data acquisition.
Pulse oximetry screening in the U.S. is recommended after 24 hours of life,Reference Diller, Kelleman, Kupke, Quary, Kochilas and Oster13 with a goal of identifying patients with hypoxemia with low false-positive results, before newborn discharge, which is typically 48 hours after birth. Infants with abnormal screening results must be referred to a physician, who evaluates the causes of hypoxemia. If there is a lack of evidence to explain hypoxemia, further diagnostic echocardiograms are required at the recommended time of screening. As the newborn patients are too young to be discharged for outpatient diagnostic echocardiograms, diagnostic echocardiograms due to abnormal screening results are included in hospital discharge records. The number of live births per month and year in each of the states was obtained using Centers for Disease Control and Prevention’s Wide-Ranging Online Data for Epidemiologic Research.14
Identification of patients
We identified infants admitted to the hospital 0–3 days after birth, who had diagnostic echocardiograms (International Classification of Diseases, Ninth Revision, Clinical Modification procedure code 88.72) 1–3 days after birth. Newborns who had diagnostic echocardiograms within 24 hours of birth were not included. As the pulse oximetry screening guidelines recommend performing the screening after 24 hours of life, changes in rate of the diagnostic echocardiograms performed within 24 hours of life were not driven by the implementation of the pulse oximetry screening. Examples of diagnostic echocardiograms performed within 24 hours of life include echocardiograms based on results from a prenatal screening or clinical exam or for the screening of a preterm infant, rather than results from the pulse oximetry screening.
Using the diagnostic codes listed in Supplemental Table S1, we identified whether a critical CHD diagnosis had been made for each patient. We defined patients who underwent diagnostic echocardiograms 1–3 days after birth, but were discharged without a critical CHD diagnosis as patients with critical CHD-negative Echo. The rationale is that referral for diagnostic echocardiograms by healthcare workers will likely occur shortly after abnormal pulse oximetry screening results, with the rate of echocardiograms occurring 1–3 days after birth, reflecting the impact of mandatory screening implementation.
Measurements
All rates were per 1000 live births in the associated states.
The primary outcome of interest was rate of patients with critical CHD-negative Echo. The secondary outcome was rate of transfers for patients with critical CHD-negative Echo. The tertiary outcome was the number of patients admitted to hospitals within 3 days of birth with critical CHD diagnoses, which provides an estimate of critical CHD cases detected before newborn discharge. Finally, we evaluated the rate of diagnostic echocardiograms for patients diagnosed with critical CHD to assess accuracy and completeness of coding for diagnostic echocardiograms in administrative databases.14 Clinically, all infants admitted with critical CHD should receive an echocardiogram; therefore, we will use the percentage of critical CHD patients admitted within 3 days of birth without an echocardiogram procedure code in their inpatient record as an estimate for the rate of missing echocardiogram codes. The diagnostic echocardiograms for newborns with critical CHD could be performed anytime; more specifically, newborns diagnosed with critical CHD prenatally are likely to be performed immediately after birth, and many critical CHDs were detected prenatally, and the last analysis included the diagnostic echocardiograms performed within 24 hours of birth.
We defined pre- and post-mandate cohorts based on the dates when pulse oximetry screening became mandated in each state. Detailed information on the enrolment period for each state is provided in Supplemental Table S2 and Supplemental File S1.
Patients with critical CHD-negative Echo included both newborns who were suspected to have heart diseases based on symptoms detected during physical examinations and newborns with abnormal pulse oximetry screening results. Cases with abnormal physical examinations will be included in both periods, whereas cases with abnormal pulse oximetry screening results are more likely to be included in post-mandate cohorts.
Statistical analyses
Patient age, sex, race/ethnicity, insurance type, transfer status (indicating patient transfer into hospital from another facility), patient residential location, and median household income for patient zip code were descriptively assessed for each period.
The rate of patients with critical CHD-negative Echo, the rate of transfers among patients with critical CHD-negative Echo, and the rate of patients who were admitted to the hospital with a critical CHD diagnosis within 3 days of birth calculated based on total number of live births in the corresponding months and years were measured. A stratified analysis by race/ethnicity (Non-Hispanic White and Non-White) based on maternal race and ethnicity was performed to assess racial disparities.
Linear segmented regression models for interrupted time series were used to assess association between implementation of state mandatory screening and changes in number of patients with critical CHD-negative Echo, transfers among patients with critical CHD-negative Echo, and patients who were admitted to the hospital with a critical CHD diagnosis within 3 days of birth.
Details for statistical models are provided in Supplemental File S2. Analyses were conducted using SAS version 9.4 (Research Triangle Institute, Research Triangle Park, NC).
Stata SE version 14.0 was used to produce the figures.
Sensitivity analyses
Pulse oximetry detects hypoxemia; therefore, even if newborns with abnormal pulse oximetry screening results did not have critical CHD, abnormal results do not necessarily indicate a false positive, but could potentially reveal other underlying conditions, indicating hypoxemia that might, otherwise, not be detected as early without pulse oximetry screening. Therefore, we conducted the following two sensitivity analyses. First, diagnostic echocardiograms could be performed not only for infants suspected of having critical CHD, but also for infants suspected of having any CHD. Pulse oximetry measurement can be abnormal for certain CHDs, so we excluded patients with any CHD diagnosis (International Classification of Diseases, Ninth Edition, Clinical Modification codes: 745–747). Second, pulse oximetry screening identifies newborns who present as hypoxemic for reasons unrelated to critical CHD or CHD, such as pneumonia. Therefore, the second sensitivity analysis excluded patients who had sepsis (International Classification of Diseases, Ninth Edition, Clinical Modification codes, Sepsis: 995.9; Bacteremia: 790.7; Septic Shock: 785.52), pneumonia (International Classification of Diseases, Ninth Edition, Clinical Modification codes: 481–486), or pulmonary hypertension (International Classification of Diseases, Ninth Edition, Clinical Modification codes: 416.0), in addition to any CHD. Finally, we excluded patients admitted to hospitals outside their state of residence, because some patients could have transferred from a state without mandatory pulse oximetry screening.
The Institutional Review Board at The Lundquist Institute for Biomedical Innovation at Harbor-UCLA Medical Center approved this study under the “exempt” category.
Results
We identified 5967 cases with critical CHD-negative Echo (2847 in the pre-mandate periods and 3120 in the post-mandate periods). Demographics in the pre- and post-mandate periods were similar, e.g., more likely male, White, on Medicaid, and residing in urban settings. The two groups had similar income status (Table 1).
Descriptive statistics of hospitalised infants who received diagnostic echocardiograms not resulting in critical CHD diagnoses at pre- and post-mandate periods
a 0-day old infants were assumed referred for echocardiogram due to antenatal diagnosis or clinical exam at birth, and not critical CHD screening by pulse oximetry (POx), which should be done no sooner than 24 hours after birth.
The rate of patients with critical CHD-negative Echo increased from 3.11 to 3.51 per 1000 live births (increased by 0.4). We found rates of patients with critical CHD-negative Echo to be higher overall among Non-White (6.35 in pre- and 7.49 in post-mandate periods) compared to White patients (1.70 in pre- and 1.71 in post-mandate periods). The rate of transfers remained relatively stable (0.19 in pre- and 0.21 in post-mandate periods). The rate of patients who were admitted to the hospital within 3 days of birth with a critical CHD diagnosis increased from 1.58 to 1.68 (Table 2).
Ratea of patients who had echocardiograms but did not receive a diagnosis of critical CHD, rate of transfer among Critical CHD-negative echocardiogram, and rate of patients who were admitted to the hospital within 3 days of birth with a critical CHD diagnosis
a Rate was per 1000 live births in the corresponding states, months, and years.
Only 31.1% of these patients have an echocardiogram procedure code in their records. Thus, 68.9% of critical CHD patients are missing an echocardiogram procedure code, most likely due to a coding error.
Our interrupted time series models detected a statistically significant increasing trend in rate of critical CHD-negative Echo in the pre-mandate period (Incidence Rate Ratio: 1.08, 95% Confidence Interval 1.01–1.16, p = 0.02) and did not detect any statistical differences in changes between pre- and post-mandate periods (Incidence Rate Ratio: 0.93, 95% Confidence Interval 0.84–1.03, p = 0.14). Stratified analyses by White and non-White suggest that the increasing trend overall can be primarily attributed to the increasing trend among non-Whites (Incidence Rate Ratio: 1.12, 95% Confidence Interval 1.05–1.19, p < 0.01). Among non-Whites, we detected a statistically significant difference in time trends between pre- and post-mandate periods for the rate of newborns who received a critical CHD-negative Echo result. The increasing trend in the pre-mandate period was attenuated during the post-mandate period (Incidence Rate Ratio: 0.89, 95% Confidence Interval 0.80–0.98, p = 0.02). Supplemental Figure S1 and Figure 1 illustrate the trend in quarterly rates of critical CHD-negative Echo in general and among non-Whites, respectively.
Quarterly hospitalisation rate of critical CHD-negative Echo* among non-White**. *: Quarterly hospitalisation rate of critical CHD-negative Echo was defined as number of patients who underwent diagnostic echocardiograms 1–3 days after birth, but were discharged without a critical CHD diagnosis divided by number of live births in a given quarter. **: Non-White was defined based on maternal race and ethnicity.
We did not detect statistically significant differences in changes in other outcomes between pre- and post-mandate periods (Table 3).
Results from linear segmented regression models of interrupted time series to evaluate the association with implementation of mandatory screening policy.
a Incidence Rate Ratio.
b Confidence Interval.
c Time refers to quarters from an initial point, which is January–March in 2008.
d Post is defined as a function that equals 1 during post-mandate period and is otherwise equal to 0.
e post×time is an interaction term between z and time.
f Pulse oximetry.
In the sensitivity analyses, when we excluded patients who had diagnostic echocardiograms but were diagnosed with any form of CHD, the sample size decreased to 1541, which means, among 5967 cases with critical CHD-negative Echo, 4426 cases had CHD diagnoses. We detected a statistically significant increasing trend in rate of newborns with CHD-negative Echo in the pre-mandate period (Incidence Rate Ratio 1.18, 95% Confidence Interval 1.10, 1.26, p < 0.01). We also detected a statistically significant attenuation in this increasing trend after post-mandate periods (Incidence Rate Ratio 0.84, 95% Confidence Interval 0.76, 0.93, p < 0.01) (Fig 2). Stratified analyses by White and non-White both mirror our overall results (Table 3).
Quarterly hospitalisation rate of CHD-negative Echo*. *: Quarterly hospitalisation rate of CHD-negative Echo was defined as number of patients who underwent diagnostic echocardiograms 1–3 days after birth, but were discharged without any CHD diagnosis divided by number of live birth in a given quarter.
When excluding those with CHD and other conditions associated with hypoxemia, the sample size decreased to 1522, which means, among 5967 cases with critical CHD-negative Echo, 4445 cases had a CHD diagnosis or other condition associated with hypoxemia. The results were similar to the first sensitivity analysis.
When excluding patients who were admitted to hospitals outside their state of residence, the results did not change substantially from the main analyses.
Discussion
Using the discharge databases from multiple states covering 18% of live births in the U.S., we investigated trends in critical CHD-negative diagnostic echocardiograms, which likely represent false-positive critical CHD screening results. We did not detect statistically significant associations between implementation of mandatory policies for pulse oximetry screening and changes in trends in critical CHD-negative diagnostic echocardiograms.
Although Abouk et al Reference Abouk, Grosse, Ailes and Oster15 showed the improved infant survival rate with CHD after the implementation of the mandatory screening policy using the nationwide database, the existing literature reporting false positive critical CHD screening results used the database from a single state.Reference Klausner, Shapiro, Elder, Colson and Loyal16,Reference Garg, Braun and Knapp17 In addition, these previous studies only presented false-positive results from pulse oximetry screening and did not demonstrate benefits and drawbacks to adding pulse oximetry screening to the routine physical examination. Currently, there is no large, uniform, comprehensive database to conduct multistate evaluation. To the best of our knowledge, this is the first study to compare changes in screening performance before and after pulse oximetry screening mandates, i.e., clinical examination alone versus pulse oximetry screening plus clinical examination. Our innovative approach to utilising administrative databases enabled us to conduct multiple state analyses and compare screening performance both before and after implementation of pulse oximetry screening policies. Our results suggest that the implementation of a mandatory screening policy does not result in an increased use of diagnostic echocardiograms on newborns without critical CHD, which could be a proxy for false-positive results from the screening. Moreover, we detected a statistically significant attenuation in an increasing trend for diagnostic echocardiograms for newborns without CHD or other secondary conditions that could cause hypoxemia. We speculate that the implementation of the mandatory screening policy might contribute to the decrease in diagnostic echocardiograms, which might not have a clinical indication.
Another novel finding of our study was an increasing trend in the rate of critical CHD-negative Echo among non-Whites during the pre-mandate period, which attenuated after the implementation of the mandatory screening policy. One possible explanation is that cyanosis may be more difficult to detect by physical examination among non-Whites than Whites due to darker skin colour. Before pulse oximetry screening, physical examinations were used to detect CHD through markers, such as cyanosis, murmurs, and poor pulses. Physicians aware of the greater difficulty in detecting cyanosis among infants with darker skin may have been more likely to send infants with darker skin for echocardiograms when results from the physical examination were ambiguous. Although this strategy would have helped prevent missed diagnoses, it would also have increased the number of non-White infants who received a critical CHD-negative Echo. As previous literature argued, knowledge of results from pulse oximetry screening could affect how physicians interpret physical examinations.Reference Knowles, Griebsch, Dezateux, Brown, Bull and Wren18,Reference Roberts, Barton, Auguste, Middleton, Furmston and Ewer19 Thus, the increasing trend in critical CHD-negative Echoes would have been attenuated after the introduction of mandatory pulse oximetry screening, as physicians relied more on the results of pulse oximetry screening instead of automatically sending non-White infants with ambiguous physical examination results for an echocardiogram.
In some of the early studies,Reference Cahan, Decker, Hoekje and Strohl20–Reference Adler, Hughes, Vivilecchia and Camargo24 it had been reported that pulse oximetry overestimates oxygen saturation in dark-skinned infants. As the equipment improved, pulse oximetry screening was still reported to overestimate oxygen in darker skin colours as high as 2.6%, but at blood oxygen concentrations lower than 80%.Reference Feiner, Severinghaus and Bickler25 As the relevant test values for infant screening are above 90% measurement, this bias would not produce higher false-negative rates. As mentioned, results from pulse oximetry screening sometimes affect how physicians interpret physical examinations, so false-negative results from pulse oximetry screening could also lead to false-negative results from physical examination.Reference Knowles, Griebsch, Dezateux, Brown, Bull and Wren18,Reference Roberts, Barton, Auguste, Middleton, Furmston and Ewer19 However, our regression results did not detect any statistically significant change in the rate of infants who were admitted to the hospital with a critical CHD diagnosis within 3 days of birth, which suggests no change in the trend for false-negative rates. Further research is needed to evaluate the effects of pulse oximetry screening results on the results of physical examination, which is beyond the scope of this study.Reference Knowles, Griebsch, Dezateux, Brown, Bull and Wren18,Reference Roberts, Barton, Auguste, Middleton, Furmston and Ewer19
We found racial disparities, especially in rates of newborns with a critical CHD-negative Echo (1.70 in pre- and 1.71 in post-mandate among White; 6.35 in pre- and 7.49 in post-mandate among non-White). Racial disparities in healthcare access and quality, including newborn screening performance, have been reported.Reference Gavin, Adams, Hartmann, Benedict and Chireau26 Racial disparities in critical CHD have also been noted, with higher mortality rates documented among Black children.Reference DiBardino, Pasquali and Hirsch27–Reference Nembhard, Salemi, Ethen, Fixler, DiMaggio and Canfield29 The current body of the literature on racial disparities in pulse oximetry screening is limited, and further research is needed to clarify underlying mechanisms behind these findings.
This study has several potential limitations. The major limitation of the study is our use of an administrative database. Although our main aim is to evaluate change in false-positive rates, identifying patients who received diagnostic echocardiograms without critical CHD diagnoses using an administrative database might not fully capture all infants with false-positive results. For example, we demonstrated that only 31.1% of the patients, who were admitted within 3 days of birth with a critical CHD diagnosis, have an echocardiogram procedure code in their records. Clinically, these patients should have received an echocardiogram. Assuming the likelihood of missing an echocardiogram code is independent of having critical CHD, our estimate for the rate of infants who received a critical CHD-negative Echo represents just 31.1% of the true rate of occurrence. Therefore, the rate of critical CHD-negative Echo could increase to 6.89 and 7.78 per 1000 live births in pre- and post-mandate periods, respectively. However, non-invasive, lower-cost procedures, such as echocardiograms, are systematically underreported in administrative databases in comparison with more expensive and invasive procedures.Reference Dismuke30–Reference Haut, Pronovost and Schneider32 Most patients diagnosed with critical CHD must also undergo invasive surgical procedures, which result in much higher costs than echocardiograms. Therefore, echocardiogram procedure codes are more likely to go unreported among critical CHD patients receiving additional and comparatively more invasive and costly procedures than those without critical CHD. We believe that the true rates of critical CHD-negative echocardiograms for both the pre- and post-mandate periods are between 3.11 and 6.89 for the pre-mandate period and between 3.51 and 7.78 for the post-mandate period. Additionally, missing procedure codes will not bias our regression model, as it is likely that missing codes for echocardiograms are independent from the implementation of the mandatory screening policy. Essentially, missing codes for echocardiograms are randomly occurring across pre- and post-mandate periods and, therefore, do not affect the regression model. A prospective study across multiple sites to capture true rates of infants admitted within 3 days of birth who receive an echocardiogram to confirm a critical CHD diagnosis would provide a starting point for more accurate data.
Second, the impact of pulse oximetry screening varies by state, and our analyses were limited to six states, so our results may not be generalisable to the U.S. as a whole. For example, a Connecticut hospital study found no infants diagnosed through critical CHD screening alone, due to high success rates of prenatal detection (85%).Reference Klausner, Shapiro, Elder, Colson and Loyal16
Third, data were evaluated at an aggregate level, and therefore, ecological fallacy exists; i.e., association at the aggregate level is not always the same as association at the individual level.
Fourth, it is assumed that the hospitals in states with mandatory pulse oximetry screening policies perform the screening more often than those in states without mandatory policies. However, this may not be the case. For example, surveyed nurse managers and hospital administrators in Georgia and Wisconsin reported that daily routines did not always incorporate recommended pulse oximetry procedures as recommended by the American Academy of Pediatrics, even though Georgia and Wisconsin were among the first wave of states to implement the mandatory screening process.Reference Beissel, Goetz and Hokanson33–Reference Kemper, Mahle and Martin35 Specifically, hospital staff reported different screening times, pulse oximetry probe placement, cutoff values to determine a pass or fail, and next steps once an infant screens positive for critical CHD. Another study conducted in West Virginia in the first year following the screening mandate found that only 85% of eligible infants had pulse oximetry screening.Reference John, Phillips, Hamilton and Lastliger36 Similar to Abouk et al’s argument, our study focused on overall effects of state mandatory policies rather than screening practices, validating the study design.Reference Abouk, Grosse, Ailes and Oster15
Finally, implementation level generally improves through natural diffusion, as intervention practices circulate among healthcare professionals over time. Due to data availability constraints, our analysis was limited to infants born before 2014. Re-evaluation with additional years of data is needed.
Conclusion
We did not detect statistically significant changes in trends of critical CHD-negative Echo, defined as newborns who underwent diagnostic echocardiograms 1–3 days after birth but were discharged without critical CHD diagnoses, before and after implementation of state mandatory screening policies. However, an increasing trend detected in the pre-mandate period among non-Whites was attenuated in the post-mandate period. This same attenuated increasing trend was detected in the sensitivity analyses, which expanded the target health issues to any form of disease that could cause hypoxemia. We speculate that the implementation of the mandatory screening policy might contribute to the decrease in diagnostic echocardiograms, which might not have a clinical indication. There is a possibility that very low false-positive rates in pulse oximetry screening affected interpretation of routine physical examination results. Further research is needed to evaluate how screening results affect results of physician examination.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1047951120001742
Acknowledgements
This study utilised 2008–2010 and 2012–2014 discharge data from Maryland and New Jersey, and 2010 and 2014 discharge data from Kentucky, Michigan, New York, and North Carolina, State Inpatient Databases, Healthcare Cost and Utilisation Project, Agency for Healthcare Research and Quality.12
Financial support
Rie Sakai-Bizmark is funded by the National Institutes of Health Research Scientist Development award [NHLBI K01HL141697]. The contents of this work are solely the responsibility of the authors and do not necessarily represent the official views of the National Heart, Lung, and Blood Institute.
Conflicts of interest
Dr Chang is the founder, CEO, and majority shareholder of QT Medical. QT Medical manufactures ECG devices. Dr Chang is also the founder and CEO of NeoVative, a Research & Development company for wearable medical devices. The authors declare that there is no conflict of interest regarding the study.
