Children with CHD who have undergone surgical intervention during infancy are at high risk for developmental challenges, including language and motor delays, adaptive behaviour deficits, and learning disabilities.Reference Wernovsky 1 The American Heart Association published a statement in 2012 recommending periodic developmental surveillance, screening, evaluation, and re-evaluation throughout childhood to identify deficits and provide appropriate therapies and education.Reference Marino, Lipkin and Newburger 2 Although several widely used developmental screening tools demonstrate good psychometric properties for the general paediatric population, including the Ages & Stages Questionnaires, Third Edition (ASQ-3),Reference Duncan 3 – Reference Squires and Bricker 6 their psychometric properties and clinical utility have not been examined in the CHD population.
The ASQ-3 is a set of parent questionnaires that measures development in communication, gross motor, fine motor, problem-solving, and personal–social domains in children aged 1–66 months.Reference Squires and Bricker 6 The ASQ-3 has yielded mixed empirical support with select paediatric illness populations including premature birth,Reference Simard, Luu and Gosselin 7 – Reference Skellern, Rogers and O’Callaghan 9 neurosensory disability,Reference Yu, Hey and Doyle 10 and hypoxic–ischaemic encephalopathy.Reference Lindsay, Healy, Colditz and Lingwood 11 In the context of CHD, the ASQ-3 has been used for developmental outcomes research,Reference Brandlistuen, Stene-Larsen, Holmstrøm, Landolt, Eskedal and Vollrath 12 – Reference Sananes, Manlhiot and Kelly 15 evaluated as a predictor of parent-reported developmental concerns three years later,Reference Elbers, Macnab, McLeod and Gagnon 16 and incorporated into national collaborative quality improvement efforts – for example, the National Pediatric Cardiology - Quality Improvement Collaborative; yet, studies have not evaluated the psychometric properties or clinical utility of the ASQ-3 in this patient population. This study aimed to evaluate the ASQ-3 as a developmental screening tool in infants and toddlers with CHD by comparing developmental classifications obtained via ASQ-3 with those obtained from the gold standard of developmental classification – the Bayley Scales of Infant and Toddler Development, Third Edition (Bayley-III).Reference Bayley 17 A secondary aim was to evaluate the added utility of the ASQ-3 after accounting for known clinical risk factors for developmental disorders in the CHD population. We hypothesised that the ASQ-3 would demonstrate clinical utility for developmental screening and improve accuracy in predicting developmental delays as compared with clinical risk factors alone.
Methods
Patients and procedures
Patients were 163 children aged 4–40 months who had undergone cardiac surgery during infancy and completed at least one neurodevelopmental evaluation through the Nemours Cardiac Center’s developmental follow-up programme between 2012 and 2014. Neurodevelopmental evaluations were administered at the approximate ages of 6, 12, 24, and/or 36 months with the Bayley-III and ASQ-3. Parents also completed a brief socio-demographic questionnaire. Select clinical variables were extracted from the medical record, including gestational age at birth, confirmed or suspected genetic syndromes, single-ventricle versus two-ventricle cardiac anatomy, age at first cardiac surgery, length of hospital stay, and cumulative lifetime duration of cardiopulmonary bypass and deep hypothermic circulatory arrest. This study was reviewed and approved by the Nemours Institutional Review Board with waiver of the requirement for obtaining informed consent/parental permission.
Measures
Bayley-III
The Bayley-III assesses developmental functioning of children aged 1–42 months through structured play activities. Scaled scores (mean of 10, standard deviation of 3) for cognitive, receptive communication, expressive communication, fine motor, and gross motor development are calculated from raw scores, and composite scores (mean of 100, standard deviation of 15) for cognitive, language, and motor development are calculated from scaled scores. Psychometric properties are well defined and acceptable.Reference Bayley 17 The Bayley-III was administered by a clinical psychologist or a psychology postdoctoral fellow. An interpreter was used for Spanish-speaking families. The Bayley-III adaptive behaviour and social–emotional parent questionnaires were not utilised as part of the neurodevelopmental evaluation, and were therefore not included in this study.
ASQ-3
The ASQ-3 is a set of parent-report, age-specific developmental screening questionnaires assessing communication, gross motor, fine motor, problem-solving, and personal–social skills.Reference Squires and Bricker 6 Each age-specific questionnaire has 40 items: six items per area and 10 open-ended questions. Response options for closed-ended questions are “yes” when the behaviour is present (10 points), “sometimes” when the behaviour is emerging (five points), or “not yet” when the behaviour is absent (0 points). Points within each developmental area are summed, and the resulting scores are compared against screening cut-off scores corresponding to 1 (“Monitor”) and 2 (“Refer”) SD below the normative mean. The ASQ-3 requires 5–10 minutes to complete and a sixth-grade reading level. The Spanish version of the ASQ-3 was utilised when appropriate. Parents completed the ASQ-3 without examiner input during each neurodevelopmental evaluation visit, following administration of the Bayley-III.
Data analyses
Receiver operating characteristic analyses using a 2x2 pass/fail design were conducted to determine sensitivity, specificity, positive predictive value, and negative predictive value of the ASQ-3 in identifying cognitive, language, fine motor, and gross motor developmental delays as measured by the Bayley-III. Positive predictive value refers to the probability that screening failure reflects true delay while negative predictive value refers to the probability that a score in the normal range reflects true normative development. ASQ-3 areas were compared with Bayley-III domains measuring similar developmental skills: problem-solving area of the ASQ-3 with the cognitive scaled score of the Bayley-III, communication area with the language composite score, fine motor area with the fine motor scaled score, gross motor area with the gross motor scaled score. The Bayley-III language composite was used rather than receptive and expressive communication scaled scores, as the ASQ-3 does not screen for receptive and expressive communication delays separately. The personal–social area of the ASQ-3 was not included in the analyses, as Bayley-III parent questionnaires measuring similar constructs such as adaptive behaviour and social emotional functioning were not administered.
The first set of receiver operating characteristic analyses defined screening failure on the ASQ-3 as ⩾1.0 SD below the normative mean, denoting a score within the “Monitor” or “Refer” ranges.Reference Squires, Twombly, Bricker and Potter 18 A second set of receiver operating characteristic analyses utilised a more stringent definition of screening failure: ⩾2.0 SD below the normative mean, denoting a score within the “Refer” range. Developmental delay as measured by the Bayley-III was defined as ⩾1.0 SD below the normative mean (scaled score ⩽7, composite score ⩽85) for all analyses. The decision to utilise a broad definition of developmental delay, capturing both children with a mild or emerging delay as well as children exhibiting severe delay, was made by considering the purpose of developmental screening for identifying children who would benefit from formal evaluation but may not be identified through surveillance alone. In this patient population, mild-to-moderate developmental delays are quite common and warrant parent anticipatory guidance, recommendation for home activities, and/or early intervention.Reference Brosig, Butcher and Butler 19 Receiver operating characteristic analyses were performed separately for each time point – 6, 12, 24, and 36 months – and by using aggregated data across all time points.
Logistic regression models assessed the utility of the ASQ-3 for predicting developmental delays after accounting for a set of known clinical risk factors in the CHD population.Reference Marino, Lipkin and Newburger 2 Length of hospital stay for first surgical admission and presence versus absence of premature birth, single-ventricle anatomy, and confirmed or suspected genetic syndrome were entered in block 1 of each model, and ASQ-3 screening failure defined as ⩾1.0 SD below the normative mean was entered in block 2 – that is, one model for each of the four developmental domains. Patients who had not undergone cardiopulmonary bypass were excluded. To reduce the number of analyses and risk for type II error, logistic regression analyses were conducted for the 12-month time point only as it provided the largest sample size.
Results
Sample characteristics
A total of 250 neurodevelopmental evaluations (ASQ-3 and Bayley-III) from 163 patients were included (6 months: n=59; 12 months: n=90; 24 months: n=69; 36 months: n=32). The mean ages at each evaluation were as follows: 6.4±1.0 months, 12.9±1.5 months, 24.3±2.2 months, and 36.5±1.5 months. Among all, 90 patients completed one evaluation, 59 completed two, and 14 completed three. Socio-demographic, clinical, and neurodevelopmental characteristics of the patients are displayed in Table 1. Bayley-III motor composite scores tended to be lower than cognitive and language composite scores across time points.
Patient characteristics and neurodevelopmental outcomes.
1Q=first quartile; 3Q=third quartile; Bayley-III=Bayley Scales of Infant and Toddler Development, Third Edition; CICU=cardiac ICU; CPB=cardiopulmonary bypass; DHCA=deep hypothermic circulatory arrest
* Missing data for 19 patients (11%)
ASQ-3 screening failure defined as ⩾1.0 SD below the normative mean
Table 2 displays area under the curve, sensitivity, specificity, positive predictive value, and negative predictive value for each ASQ-3 area with ASQ-3 screening failure defined as ⩾1.0 SD below the normative mean. Across all developmental areas, specificity was excellent (⩾81.9%), and the probability that a score in the normal range reflected true normative development was high (negative predictive value ⩾81.0%). Sensitivity and positive predictive value varied by developmental area. The gross motor area demonstrated excellent sensitivity (79.6%) in identifying children with delay, and screening failure was associated with a high probability of true delay (positive predictive value=84.9%). Screening failure in the communication area was also associated with a high probability of true delay (positive predictive value=81.4%), but its sensitivity was lower (67.8%). The problem-solving and fine motor areas demonstrated only fair sensitivity and relatively low positive predictive value (<70%).
Area under the curve (AUC).
ASQ-3=Ages and Stages Questionnaires, Third Edition; n=number of assessments; NPV=negative predictive value; PPV=positive predictive value
Examination of the ASQ-3 at individual time points (Supplementary Table 1) revealed that sensitivity increased with age for the communication area (6 months=35.7%; 12 months =74.3%; 24 months =90.0%, 36 months=100.0%) and decreased between 12 and 36 months of age for the problem-solving area (12 months=73.1%; 24 months=66.7%, 36 months=50.0%) (Fig 1).
Sensitivity of Ages and Stages Questionnaires, Third Edition (ASQ-3) areas by time point. *ASQ-3 screening failure defined as ⩾1.0 SD below the normative mean.
ASQ-3 screening failure defined as ⩾2.0 SD below the normative mean
Table 2 displays the area under the curve, sensitivity, specificity, positive predictive value, and negative predictive value using a more stringent definition of screening failure: ⩾2.0 SD below the normative mean. Specificity was excellent across all areas (⩾93.6%). Sensitivity values, however, were low to fair, ranging from 31.1% (communication) to 62.8% (gross motor). Screening failure was associated with a high probability of true delay for the communication, fine motor, and gross motor areas (positive predictive value ⩾85.1%), but with a somewhat lower probability of true delay for the problem-solving area (positive predictive value=74.5%). The probability that a score in the normal range reflects true normative development was high for the problem-solving area (negative predictive value=81.6%), but lower for the communication, fine motor, and gross motor areas (negative predictive value ranging from 68.7 to 75.0%).
The probability that a score in the normal range reflects true normative development increased with age for the communication area (negative predictive value: 6 months=53.9%; 12 months=64.5%; 24 months =84.0%; and 36 months=85.0%) (Supplementary Table 2).
Added utility of ASQ-3 beyond clinical risk factors
After accounting for known clinical risk factors for developmental disorders, the ASQ-3 significantly predicted Bayley-III developmental classifications at 12 months of age. Compared with the model with clinical risk factors only, the addition of the ASQ-3 increased the percentage of correctly classified patients from 72.3 to 80.7% for the Bayley-III cognitive domain, 68.8 to 87.5% for language, 71.4 to 78.6% for fine motor area, and 76.2 to 86.9% for gross motor area (Table 3). ASQ-3 screening failure in problem-solving, communication, fine motor, and gross motor areas was associated with odds ratios of 7.65 (95% CI=2.33–25.15), 209.20 (14.49–3020.20), 3.82 (1.29–11.33), and 49.96 (8.05–309.94), respectively, for developmental delay in corresponding Bayley-III domains.
Percentage of correctly classified patients at 12 months.
ASQ-3=Ages and Stages Questionnaires, Third Edition; Bayley-III=Bayley Scales of Infant and Toddler Development, Third Edition
* Delay defined as ⩾1.0 SD below the normative mean
** Includes premature birth, single-ventricle anatomy, confirmed/suspected genetic syndrome, and length of hospital stay (first surgical admission)
*** Includes ASQ-3 delay defined as ⩾1.0 SD below the normative mean and clinical variables
Discussion
Although standardised developmental screening tools are important for the evaluation and management of developmental disorders in children with CHD, this is the first study to our knowledge examining the psychometric properties and clinical utility of a specific developmental screening tool in this population. We have shown that the ASQ-3 has clinical utility for screening development in infants and toddlers with CHD and improves the accuracy in predicting delays over clinical risk factors alone.
The ASQ-3 appears particularly useful for screening gross motor development, with 80% of mild-to-severe gross motor delays and 88% of typical development identified using the 1 SD cut-off. This is in contrast to a previous investigation of preterm infants in which the ASQ-3 failed to accurately discriminate patients with motor delays.Reference Simard, Luu and Gosselin 7 Although these varied results may be partially attributable to methodological differences between the studies – Second versus Third Edition of the Bayley Scales or mode of ASQ-3 administration – it is possible that the ASQ-3 has increased utility in the gross motor domain for infants and toddlers with CHD as compared with preterm infants. A developmental screening tool with high sensitivity and specificity in the area of gross motor is relevant for this population, as gross motor delays are among the most common developmental challenges following infant heart surgery.Reference Mussatto, Hoffmann and Hoffman 20 , Reference Newburger, Sleeper and Bellinger 21 Furthermore, the ASQ-3 demonstrated clear benefit over a review of risk factors alone for predicting gross motor delay. It is not known how the ASQ-3 gross motor area would fare in comparison with routine physician-based surveillance and examination, and this should be explored in future studies.
The clinical utility of the ASQ-3 for screening cognitive and language development varied by time points, similar to ASQ-3 findings reported with hypoxic–ischaemic encephalopathyReference Lindsay, Healy, Colditz and Lingwood 11 and preterm birth.Reference Simard, Luu and Gosselin 7 The ASQ-3 appears useful for screening cognitive development in infants but not in toddlers, with only 67 and 50% of cognitive delays identified in the 24- and 36-month samples, respectively. Some items used to assess cognitive/problem-solving development on the ASQ-3 at 24 and 36 months of age may not be commonly observed in the home environment – for example, “Show your child how to make a bridge with blocks […] Does your child copy you by making one like it?” – when compared with 6 months of age – for example, “Does your baby play by banging a toy up and down on the floor or table?” – and therefore may not be as accurately reported. Conversely, the ASQ-3 appears useful for screening language development in toddlers but not in infants. Prelinguistic communication in infancy, such as responsiveness to sounds and non-word vocalisations/gestures, may be particularly susceptible to subjective interpretation. Clinicians should use caution when interpreting problem-solving and communication results for these age groups and consider supplementing the ASQ-3 with parent interview and behavioural observations.
The ASQ-3 has two screening cut-off scores corresponding to 1 SD – “Monitor” – and 2 SD – “Refer” – below the normative mean. Although continuing to monitor children from the general population who score between 1 and 2 SD below the normative mean may be appropriate, children with CHD exhibit a pattern of low-severity or combined developmental disabilities that can impact functioning if not addressed.Reference Wernovsky 1 , Reference Mussatto, Hoffmann and Hoffman 20 Recognition of this pattern has led clinicians and researchers to recommend formal developmental evaluation and intervention even in the case of mild delays.Reference Marino, Lipkin and Newburger 2 , Reference Mussatto, Hoffmann and Hoffman 20 Results of the present study suggest that referring only children whose scores fall ⩾2 SD below the normative mean for evaluation would result in missing as many as 69% of children with a mild-to-severe delay, although the rate of false positives would decrease from 10 to 18% with the 1 SD cut-off to as low as 1–6% with the 2 SD cut-off. As with any screening tool, the optimal cut-off score depends on a balance of sensitivity and specificity dependent on risk associated with over- or under-identifying the condition. Formal developmental evaluation has no physical risks and is often helpful even in the absence of significant delay, as it is an opportunity to provide parent education/consultation and make recommendations for home activities to promote developmentReference Brosig, Butcher and Butler 19 and could prevent more significant problems from developing. Therefore, clinicians are encouraged to refer children scoring ⩾1 SD below the normative mean on any ASQ-3 area for developmental evaluation.
The positive predictive value and negative predictive value provide important information that clinicians can reference when counselling families about a child’s risk for delay and need for formal evaluation. Children scoring at or below the 1 SD cut-off for the gross motor and communication areas had 85 and 81% probability of true delay, respectively, and those scoring at or below the 2 SD cut-off for these same areas had 94 and 93% probability of true delay, respectively. Therefore, clinicians can be confident in their referral for evaluation in the context of low ASQ-3 gross motor and communication scores. The probability of true delay among children scoring 1 or 2 SD below the normative mean on the problem-solving or fine motor areas of the ASQ-3 was somewhat lower, likely because these delays are less obvious to parents and have a lower base rate in this patient population during infancy/toddlerhood.Reference Sananes, Manlhiot and Kelly 15 , Reference Mussatto, Hoffmann and Hoffman 20
The probability that a score in the normal range reflects true normative development ranged from 81 to 87% with the 1 SD cut-off and from 69 to 82% with the 2 SD cut-off, underscoring the importance of periodic developmental evaluation and re-evaluation for children considered high risk for developmental delay based on medical history, regardless of screening results.Reference Marino, Lipkin and Newburger 2 Basing referral for evaluation on a screening tool alone would likely result in missing a number of children who may be experiencing subtle challenges that warrant consultation and intervention. Similarly, a screening tool cannot replace formal evaluation, even when a clinician suspects delay based on screening results. Comprehensive evaluation is necessary to identify strengths and weaknesses, provide recommendations, and develop treatment plans.Reference Marino, Lipkin and Newburger 2 , Reference Brosig, Butcher and Butler 19
The added utility of the ASQ-3 to risk factor analysis is supportive of a model of combined developmental surveillance – including risk factor review – and screening. Although many paediatric cardiac centres now offer developmental evaluation as part of routine care for children with CHD,Reference Brosig, Butcher and Butler 19 this is not yet universal. The availability of a brief, validated screening tool that can be completed during a medical visit may be particularly helpful in the absence of an on-site developmental follow-up programme. Individualised information about a child’s development – for example, positive predictive value and negative predictive value data indicating a child’s risk of delay – can be used to counsel families on the importance of formal developmental evaluation, which may be more compelling to parents than clinical risk factors alone.
This study utilised data from a single institution’s developmental follow-up programme, which serves predominantly patients who are at high risk for developmental disorders, precluding separate examination of ASQ-3 utility in low-risk patients. The ASQ-3 was not administered at the specific time points recommended by the American Heart Association guidelines, as the structure of this developmental follow-up programme predated publication of the guidelines. The time points, however, do align with other national collaborative efforts to implement ASQ-3 screening within this patient population – for example, National Pediatric Cardiology - Quality Improvement Collaborative. Future studies should evaluate the clinical utility of the ASQ-3 at time points recommended by the American Heart Association guidelines, with separate analyses for subgroups known to be at varying risk for developmental delays, such as children with CHD within the context of a genetic syndrome, those with single-ventricle anatomy, or those classified as low-risk for developmental disorders on the basis of medical history. Most children completed one or two evaluations during the period of this study, resulting in a different subset of children for each time point. Future studies are needed to evaluate performance trends as well as changes in ASQ-3 psychometric properties and clinical utility longitudinally with the same cohort. Finally, the personal–social area of the ASQ-3 was not examined in this study, but should be evaluated in future studies, suggesting impairments in adaptive behaviour and social cognition in this patient population.Reference Marino, Lipkin and Newburger 2
Conclusion
The ASQ-3 demonstrates utility for the developmental screening of infants and toddlers with CHD and provides additional predictive accuracy above and beyond clinical risk factors. Additional studies are needed to evaluate ASQ-3 psychometric properties and clinical utility when administered longitudinally at multiple congenital heart centres to parents of children with varying CHD severity and co-morbidities. A developmental screening tool validated for this patient population provides the opportunity to identify developmental concerns early and refer for formal evaluation and to carry out interventions to minimise the impact on quality of life.
Acknowledgements
The authors thank the families who participated in the Nemours Cardiac Center’s developmental follow-up programme, from which these data were obtained.
Financial Support
Dr Sood’s work on this project was supported by an Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under grant number U54-GM104941 (PI: Binder-Macleod).
Conflicts of Interest
None.
Ethical Standards
The authors assert that all procedures contributing to this work comply with the ethical standards of national guidelines on human experimentation and with the Helsinki Declaration and have been approved by the Nemours Institutional Review Board.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1047951117000415
