Introduction
Neurodevelopmental problems and CHD
CHD is the most common major birth defect, affecting 6–8/1000 live births.Reference Van Der Linde, Konings and Slager 1 Improvements in medical and surgical management have resulted in a growing population of children and adults living with CHD and its consequences, including those resulting from cardiac surgery; paradoxically, improved survival, particularly of those with more complex CHD, translates into a greater number of children and young people living with significant neurodevelopmental morbidity.Reference Mussatto, Hoffmann and Hoffman 2 – Reference Snookes, Gunn and Eldridge 4
Neurodevelopmental abnormalities are the most frequent morbidity in survivors of cardiac surgeryReference Gaynor, Stopp and Wypij 5 and include motor deficits,Reference Long, Harris, Eldridge and Galea 6 – Reference Kirshbom, Flynn and Clancy 9 seizures,Reference Bellinger, Jonas and Rappaport 10 – Reference Rappaport, Wypij and Bellinger 13 poor executive functioning,Reference Marino, Beebe and Cassedy 14 communication problems,Reference Hövels-Gürich, Konrad and Skorzenski 8 , Reference Bellinger, Wypij and Duplessis 15 , Reference Mahle, Clancy, Moss, Gerdes, Jobes and Wernovsky 16 impairments in visual construction and perception,Reference Bellinger, Wypij and Duplessis 15 – Reference Miatton, De Wolf, François, Thiery and Vingerhoets 19 poor attentionReference Hövels-Gürich, Konrad, Skorzenski, Herpertz-Dahlmann, Messmer and Seghaye 20 , Reference Shillingford, Glanzman, Ittenbach, Clancy, Gaynor and Wernovsky 21 , and learning difficulties.Reference Hövels-Gürich, Konrad and Skorzenski 8 , Reference Kirshbom, Flynn and Clancy 9 , Reference Shillingford, Glanzman, Ittenbach, Clancy, Gaynor and Wernovsky 21 Deficits can range in severity and may be subtle and therefore more easily overlooked, particularly in children with less complex CHD, and throughout the course of childhood and adolescence the presentation of neurodevelopmental abnormalities can change. Some deficits may resolve spontaneously; others may not be apparent until later childhood. Recent longitudinal evaluation of a cohort of pre-school-aged children at high risk of developmental delay indicated an improving developmental trajectory in some, but approximately 20% had scores in one or more developmental domains which decreased over time.Reference Mussatto, Hoffmann and Hoffman 22 Presentation of deficits can be obscured or confounded by a range of factors, including those related to cardiac surgery, the effects of hospitalisation, and other co-morbidities. The American Heart AssociationReference Marino, Lipkin and Newburger 23 recently published guidelines for systematic surveillance, screening, evaluation, and intervention to identify neurodevelopmental problems early and optimise outcomes in the short and longer term, building on earlier guidelines from the American Academy of Pediatrics. 24 They also highlighted the importance of continued monitoring because the level of risk for neurodevelopmental impairment can change over time as different impairments become apparent during different periods of development. Furthermore, children at risk for poor late outcomes are frequently not identified from results of early testing.Reference Mcgrath, Wypij, Rappaport, Newburger and Bellinger 25 It is therefore not surprising that increasing numbers of follow-up programmes for children with CHD and neurodevelopmental concerns are now being implemented, particularly in the United StatesReference Soto, Olude and Hoffmann 26 and some countries in Europe.
The United Kingdom National Health Service context
In contrast to the United States, other countries in Europe and Australia, there have been few published manuscripts from the United Kingdom about neurodevelopmental outcomes in children with CHD. However, the United Kingdom does require all paediatric cardiac centres to submit surgical and outcomes data to a national audit database.
During the conduct of a multi-centre study incorporating the prospective standardised detection of peri-operative complications and follow-up of children for 6 months following paediatric cardiac surgery,Reference Brown, Pagel and Brimmell 27 – Reference Tsang, Anderson and Barron 29 it has been noted based on feedback from health professionals and parents that there is considerable variability in terms of follow-up of developmental delay for children with CHD. After an acute neurological event post-surgery, a child will be assessed by a neurologist but a child who has developmental delay in the absence of an acute neurological event may not be referred to a developmental specialist. Although all children with CHD are under follow-up with a paediatric cardiologist, such professionals are in general not trained to undertake developmental assessments. Children with CHD and other co-morbidities including developmental delay may be under the care of a paediatrician at their local hospital and may be referred by the cardiac centre or their paediatrician to local services based on suspicion of developmental delay, but there is seldom the opportunity or resources for any developmental testing to provide evidence to support those concerns.
Every child in the United Kingdom, irrespective of any known health problems, has an allocated health visitor – a community-based nurse or midwife – who should make a minimum of five key visits to every family from the antenatal period until the child is 2–2.5 years old. Neurodevelopmental problems in children with CHD may arise from multiple aetiologies and at all stages in their early lives and care pathwaysReference Gaynor, Stopp and Wypij 5 , Reference Long, Harris, Eldridge and Galea 6 , Reference Bellinger, Jonas and Rappaport 10 , Reference Rappaport, Wypij and Bellinger 13 ; hence, these may be an issue before the age of 2–2.5 years, thereby delaying recognition and intervention.
A formal standardised assessment of development using comprehensive tests such as the Bayley Scales of Infant Development,Reference Bayley and Reuner 30 Griffiths Mental Development scalesReference Griffiths 31 , or the Mullen Scales of Early LearningReference Mullen 32 is considered to be the gold standard, but these are not tests that are used for early recognition of developmental delay. If a problem is identified and a child is referred for follow-up, such tests are usually undertaken by someone who has had specialist developmental training and has time dedicated to perform the testing. However, if a child is not referred, such tests are unlikely to be performed as they are not integrated into the routine follow-up of children. Moreover, this would not be feasible because the tests are time-consuming and not practicable with sick children or within the context of a busy ward or outpatients. Of note, children may require multiple cardiac interventions and the level of risk may change over time, thereby necessitating some mechanism for repeated routine monitoring and early recognition of developmental problems in all children with CHD (not an inconsiderable number). Given this whole picture, and as we know anecdotally from our own clinical practice, developmental delay is often detected late – frequently not until a child starts school and education services become involved – thereby precluding the opportunity for early intervention and causing stress to families who may be aware that “something is wrong”.
Intended improvements
With the above in mind, we determined that an early recognition tool would help to address some of the current shortcomings and facilitate appropriate and timely referral for further, more comprehensive neuropsychological evaluation for those in whom this is indicated. Our aim was to identify an existing tool that would be fit for purpose or, failing that, to develop a new tool that could subsequently be validated but that would, importantly, have the potential to be routinely used within tertiary programmes managing children with CHD.
Methods
Preliminary work: review of existing tools
A multidisciplinary team of 10 experts, comprising a paediatric neurologist, developmental experts, paediatricians, psychologists, nurses, and a statistical expert (the BDA Development Group), was convened. The BDA Development Group generated a set of consensus-based criteria for the optimal early recognition tool (Table 1) in children with heart disease.
Consensus-based criteria for neurodevelopmental assessment tool
A review of the published literature was undertaken relating to children with CHD and critical illness in order to identify the range of neurodevelopmental assessment tools available. Tests of specific domains of function, e.g. the Peabody Developmental Motor ScalesReference Folio and Fewell 33 , were not included knowing that they do not assess all the relevant domains of neuropsychological functioning. The neurodevelopmental assessment tools available were evaluated against the stated pre-set criteria.
Development of the Brief Developmental Assessment
At all stages, the BDA Development Group held underpinning goals that the final version of the Brief Developmental Assessment should be valid, reliable, understood by and useful to health professionals, acceptable to parents, accessible, resource efficient, cover the relevant areas of child development for the population of CHD, and aid early referral of neurodevelopmental problems.Reference Fischer, Morris and Martines 34
Recruitment and process for developing and testing the Brief Developmental Assessment
Stage one
The BDA Development Group decided, by consensus and based on the age distribution of children undergoing paediatric cardiac surgery in the United Kingdom, on the most appropriate age groups. Having reviewed relevant literature on neurodevelopment in children with CHD and existing measures of development (see Supplementary Table 1 for references), the conceptual basis and content of the Brief Developmental Assessment were determined in terms of the domains to be assessed and individual items to be included within each domain. On completion of the initial questionnaire development (version one), a convenience sample of 50 children with a range of cardiac diagnoses and their parents were recruited from the cardiac inpatient or day-case ward at our Institution to assess feasibility and acceptability. Our approach to recruitment of the convenience samples reported in this study was that when a member of the research team was available, he/she approached all of the children with heart disease who were admitted to the cardiac inpatient or day-case ward within the specified age range who were available based on occurrence of clinical care procedures, and those families who consented to participate then did so. Each convenience sample included children considered to be at high risk for developmental disordersReference Marino, Lipkin and Newburger 23 , but it was also important that children not typically considered to be at high risk were included because of our aim of developing a tool that could be used with children with any cardiac diagnosis. Medical and nursing staff trained in the use of the Brief Developmental Assessment administered it to at least 5 children in each age group, and parents and clinicians were asked to provide verbal feedback about the process of administering the Brief Developmental Assessment and the content of the questions, which was documented in the form of contemporaneous notes. The BDA Development Group then made any necessary changes to the content, based on the feedback, and a scoring system was added (version two).
Stage two
A further convenience sample of 138 children, which included some children with a condition known to be associated with developmental delay, were recruited from the cardiac inpatient or day-case ward for pilot testing of version two and the Brief Developmental Assessment was administered by nursing or medical staff. A preliminary assessment of internal reliability was undertaken by calculating Cronbach’s alpha values. For each age band we examined reliability across all individual scale items, between domain totals, and within each domain. Construct validity was assessed by comparing scores of 17 children in a known group with an age-matched group of 17 children who were not in a known group. A Wilcoxon test for paired data was used to compare the two groups.
Medical and nursing staff members were again asked to provide verbal feedback about the content and scoring and members of the BDA Development Group met with a group of community paediatricians to receive further feedback about the presentation, scoring, and utility of the Brief Developmental Assessment, with contemporaneous notes being taken at each occasion. Further revisions were made to both the content and scoring system by the BDA Development Group, with some problem items adjusted, resulting in version three.
Stage three
A “traffic light” scoring system was developed by the BDA Development Group, based on age-appropriate norms,Reference Kliegman, Behrman, Jenson and Stanton 35 , Reference Illingworth 36 and a detailed training guide was developed. Inter-rater reliability of version three was assessed with a convenience sample of 74 children by two research assistants, one of whom administered the Brief Developmental Assessment and scored it and a second research assistant observed and independently scored the Brief Developmental Assessment based on the performance of the child and from the information provided by the parent. The Brief Developmental Assessment scores obtained by the two research assistants were statistically compared using a weighted kappa for the gross motor score (possible range was 0–4) and an intra-class correlation for the cognitive score (possible range was 0–20). In addition, a sample of 15 parents and seven clinicians who had used the Brief Developmental Assessment were informally interviewed about their experience of using it, in terms of ease of use and understanding, relevance, time taken, and suitability of using it in the ward or clinic situationReference Fischer, Morris and Martines 34 , Reference Morelli, Pati and Butler 37 and responses documented and reviewed by the BDA Development Group in order to qualitatively evaluate the acceptability and feasibility of the Brief Developmental Assessment.
The first three stages of development and preliminary testing of the Brief Developmental Assessment are summarised in Figure 1. In Stage 4 (January 2014), the BDA Development Group signed-off the final versions of the Brief Developmental Assessment in preparation for a more extensive validation study.
Flow Diagram of Methods in the development of the Brief Developmental Assessment (BDA).
Results
The review of existing measures indicated that no single existing developmental tool fulfilled all of our criteria (Supplementary material – Table 1). Several measures only failed to meet the criterion of having elements of both observational and parent report. We considered the option of adapting one of these measures, but this would have necessitated both reducing the parental component and adding in observational items for each domain.
Content and structure
The age bands were as follows: age band one, 0–16 weeks; age band two, 17–34 weeks; age band three, 35–60 weeks; age band four, 15 months–2.9 years; and age band five, 3.0–4.9 years. This age banding was decided by consensus between members of the BDA Development Group, taking into account a focus on pre-school children to facilitate early recognition of problems, the need for the age groups to adequately capture the natural trajectory of child development, which is particularly rapid in the youngest children, and the likely population of children with CHD who undergo cardiac surgery, which incorporates a large proportion of very young children.Reference Crowe, Brown and Pagel 38 As a result, age bands were skewed towards the youngest age groups.
The domains identified as being important from the review of the literature and existing developmental measures (Supplementary Table 1) determined the conceptual basis of the tool. The domains were as follows: gross motor, daily living skills, communication and socialisation for all ages, fine motor (from 35 weeks), and general understanding (for children of 17 weeks–4.9 years). Included items for each of these domains were based on existing measures of developmentReference Bayley and Reuner 30 , Reference Griffiths 31 and were selected to reflect developmental progression within an individual age band. Behaviour items were included for children aged 15 months to 5 years but were not included in the scoring.
Changes to the content and structure from version 1 to version 3 are shown in Figure 2.
Main characteristics of each version of the Brief Developmental Assessment and summary of changes made.
Scoring
Each individual domain in all age bands had a potential score between 0 and 4. Hence, the Brief Developmental Assessment gross motor domain had a maximum score of 4 for all ages. The remaining Brief Developmental Assessment domains corresponding closely to cognitive processes were grouped together. The Brief Developmental Assessment cognitive score within the youngest age band consisted of four domains and within the older four age bands consisted of five domains, hence generating a maximum total score of 16 under the age of 17 weeks and 20 between 17 weeks and 5 years.
To address expected incremental developmental achievements with increasing age, for each age band the four items within each domain were placed in order of increasing difficulty. A child at the bottom of a given age band (youngest) would be likely to complete fewer items than a child at the top of the same age band (oldest). Therefore, in addition to ordering the items, the age range for each age band was also divided into quarters, and the expected “normal” score range within each age quartile was identified (Green), as well as the lower scores determining Amber and Red. Thus, for each domain there was a clear process for translating the number of items achieved into Red/Amber/Green on the basis of the precise age of the child. (An example of this is given in the Supplementary material, Fig 1.)
Internal reliability
Internal reliability data on 138 children based on Brief Developmental Assessment version two are displayed in Table 2. Of note, certain poorly performing items (Cronbach's α <0.6) were revised in the next iteration of the Brief Developmental Assessment. For example, in age band one the motor scale was revised to include both a gross motor domain and a fine motor domain in version 3, with four items in each. The scores for age band one were the most problematic, with weak reliability, but increasingly good reliability in the older age bands.
Results of preliminary internal reliability testing based on Cronbach’s α values.
* “All individual items” refers to each individual item on the whole scale, irrespective of which domain
** Domains other than the gross motor domain were grouped together (referred to as BDA Cognitive); the Cronbach α value represents the internal reliability of the domain total scores, other than gross motor
Construct validity
The 17 children in a known group (Down syndrome: n=12, other genetic syndromes: n=5) had significantly lower Brief Developmental Assessment scores (median: 10; IQR: 7) than their age-matched counterparts (median: 16; IQR: 5.50; Z=3.08; p=0.002), with an effect size of 0.53 (equating to a medium effect size). There were at least two matched pairs in each of the five age bands; hence, individual numbers in each age band were too low for valid statistical comparison. However, there was no evident difference in the scores in the youngest age band.
Inter-rater reliability
Inter-rater reliability data on 74 children based on Brief Developmental Assessment version 3 were excellent (Table 3). In terms of inter-rater concordance for children scoring as a “Red” in each age band, there was perfect agreement on both the Brief Developmental Assessment gross motor and Brief Developmental Assessment cognitive score scales for children in each of the age bands 1–4 (κ=1; p<0.001). In age band 5, no child scored “Red” on the Brief Developmental Assessment gross motor, and on the Brief Developmental Assessment cognitive score there was again perfect agreement (κ=1; p<0.01). The numbers of children scoring as “Red” on the Brief Developmental Assessment gross motor were 11, 4, 4, 5, and 0 for age bands 1–5, respectively, and for the Brief Developmental Assessment cognitive score the numbers scoring “Red” were 4, 7, 4, 8, and 2 for age bands 1–5, respectively.
Results of preliminary inter-rater reliability testing.
BDA Cognitive: total for all subscales other than gross motor subscale
Acceptability and feasibility
-
∙ The Brief Developmental Assessment took up to 10 minutes to complete and score, unless there was a requirement to use an interpreter.
-
∙ It was feasible to undertake the Brief Developmental Assessment in a ward or clinic setting, in terms of timing, space, and integration with other ward or clinic work.
-
∙ A range of clinicians, in particular those based in community paediatric settings, were in favour of the traffic light scoring system.
-
∙ Parents responded favourably to the Brief Developmental Assessment, reporting that the content was relevant to their child and they found it acceptable and useful for their child to be assessed with it, and nurses commented that completing the Brief Developmental Assessment was a good “ice-breaker” for children at pre-admission clinics and that it helped to build rapport with the child and family.
-
∙ The importance of training with explicit instructions for each item was emphasised by clinicians.
Discussion
The motivation for this study was, first, the acknowledgement that children undergoing cardiac surgery – approximately 5500/annum in the United KingdomReference Brown, Crowe and Franklin 39 – are at significant risk for subsequent neuropsychological impairment;Reference Mussatto, Hoffmann and Hoffman 2 – Reference Snookes, Gunn and Eldridge 4 second, an awareness that the issue of developmental delay amongst children with CHD is of great importance to stakeholders, in particular patient and parent groups,Reference Pagel, Brown and Mcleod 28 and early intervention is important for the child yet surveillance of neurodevelopment is not part of routine care; and third our realisation that the National Health Service is under immense resource pressure and constraint and hence will never foreseeably be able to support a model based on the recent recommendations for specialist neurodevelopmental follow-up of all children with CHD.Reference Marino, Lipkin and Newburger 23 Rather, we hope to develop a patient pathway that takes advantage of the structure of the National Health Service and existing services for children with CHD that has a better chance of success, noting that CHD services in the United Kingdom are centralised, and neurodevelopmental services are based in secondary care settings. Therefore, it is important to be able to identify children who need further assessment but not to overburden the system by routinely referring all children. To this end, the Brief Developmental Assessment has been developed. It is important to emphasise that the Brief Developmental Assessment does not represent a full neurodevelopmental assessment; rather, it is an early recognition tool, which may be used to raise awareness of neurodevelopmental issues in CHD and trigger more timely evaluations of neurodevelopment in the most vulnerable children by appropriate practitioners. Furthermore, given that neurodevelopmental deficits may not emerge or be apparent from the outset, some deficits may “recover” spontaneously and there is also the potential for the level of risk to change over time related to clinical factors, the Brief Developmental Assessment provides a mechanism for regular screening, as recently recommended.Reference Marino, Lipkin and Newburger 23
Despite the existence of many validated tests – both those designed for comprehensive evaluation and screening tests (see Supplementary Table 1) – of neuropsychological functioning in children, we were unable to identify an early recognition tool for use in children with CHD that fulfilled criteria established by a multidisciplinary team of experts. In particular, in order to meet the demands of a busy clinical service and a high patient volume within a financially constrained system, it will be necessary for such assessments to be undertaken by nurses or junior doctors (with appropriate training) rather than by developmental specialists. The use of screening and early recognition tools by non-specialist health professionals has been reported across a wide range of measures and professional groupsReference Fischer, Morris and Martines 34 , Reference Robertson, Hatton and Emerson 40 and their use by non-specialists is considered to be acceptable, as well as practical, given the remit and requirements of such tools. Assessments need to be feasible without specialist equipment – although more generic items such as paper and crayons would be expected to be available on a paediatric ward – and need to be able to be completed and scored relatively quickly. Assessment and surveillance are required across the range of ages from birth to adolescence and particularly up to the age of five years when children start school as these are the ages at which cardiac interventions, in some cases serial interventions, are undertaken. Furthermore, assessment must incorporate the key domains of neurodevelopment which are known to be at risk in children born with CHD who may sustain brain injuries beforeReference Sun, Macgowan and Sled 41 or soon after birthReference Licht, Wang and Silvestre 42 or around the time of surgeryReference Mussatto, Hoffmann and Hoffman 2 , Reference Snookes, Gunn and Eldridge 4 and should include direct observations and not rely entirely on parental report in order to minimise bias.Reference Gaynor, Nord and Wernovsky 43 Finally, it is important that completion and scoring of a measure should provide cues for further, timely, evaluation and treatment – previously referred to as a useful guide to action.Reference Fischer, Morris and Martines 34
Development of the Brief Developmental Assessment incorporated a thorough and iterative process involving a multidisciplinary panel of experts and a review of the literature to inform item selection, thus ensuring acceptable face and content validity. A similar approach has recently been reported in the development and validation of an early childhood development scale for use in low-resourced settings.Reference Mccoy, Sudfeld and Bellinger 44 There was iterative testing with revisions informed by feedback and a strong steer for the Brief Developmental Assessment to be practical, acceptable, and feasible to use in routine clinical practice, hence the inclusion of a traffic light system to aid interpretation of the results and guide actions. As reported, preliminary testing of the Brief Developmental Assessment was encouraging, with excellent inter-rater reliability and reasonable internal consistency for most domains. Some domains in version two had low alpha coefficients, which may suggest that these were not measuring the same construct and/or may be related to the relatively small number of items in the subscales and the small numbers of children involved in the preliminary testing. On the preliminary assessment of construct validity, the Brief Developmental Assessment distinguished those in a known group from those not in a known group, but this may not hold true for the youngest-age babies as developmental delay as a result of a syndrome has not manifested. A number of issues were identified related to understanding and clarity of certain items, and to ensure consistency of administration and scoring and to minimise any ambiguity a detailed training guide was developed.
Moving forwards, it is evident that there is a need for results of the Brief Developmental Assessment to be linked to a referral pathway which is appropriate and acceptable to clinicians working in different settings. The lack of such a pathway is a clear limitation and needs to be addressed during the next stage of work. To that end, a Delphi survey involving professionals from various disciplines and across sectors in the United Kingdom is currently underway. Furthermore, the Brief Developmental Assessment, now developed, requires full formal validation with an appropriately powered sample in each age group. This has now been completedReference Brown, Ridout and Pagel 45 and includes an evaluation of the sensitivity and specificity of the Brief Developmental Assessment against the gold-standard tests.Reference Tsang, Anderson and Barron 29 Consideration also needs to be given to the impact of false negatives and false positives for families, as well as health professionals.Reference Maxim, Niebo and Utell 46 It is clearly important for an early recognition tool to have a low false negative rate, but it has also been suggested that the rate of false positives should be no more than 30%.Reference Cicchetti, Volkmar, Klin and Showalter 47 Finally, before wider implementation of the Brief Developmental Assessment, a training package will need to be developed.
Conclusion
We have reported the development and initial stages of testing of an early recognition tool for use with children with CHD: the Brief Developmental Assessment. Our aim is to fill a gap in the health surveillance of children with CHD and to address a significant and increasingly common event – namely impaired neurodevelopmental functioning – by facilitating early, targeted referral for reassessment through the implementation of an early recognition tool.
Acknowledgements
The authors would like to thank Kate Bull and Carlos D’Souza for their valuable contributions during the development of the Brief Developmental Assessment; the community paediatricians in Haringey who offered excellent advice about the presentation, scoring, and utility of the Brief Developmental Assessment; and Dora Wood, Carol Bodlani, Aisling Bunting, Aarti Patel, and Rachel McKail who undertook the preliminary testing.
Financial Support
K.L.B., D.R., A.H., J.W., L.S., A.S., and S.O.C. were supported by the National Institute for Health Research Biomedical Research Centre at Great Ormond Street Hospital for Children NHS Foundation Trust and University College London. Professor M.L. was supported by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care North Thames at Bart’s Health NHS Trust. The views expressed are those of the author(s) and not necessarily those of the National Health Service (NHS), the National Institute for Health Research, or the Department of Health.
Conflicts of Interest
None.
Ethical Standards
The project was reviewed by, and registered with, the Great Ormond Street Hospital Clinical Audit team who approved it as a quality improvement project. Parents of children who were directly involved in testing received written information and provided verbal consent for their child’s participation.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1047951117002918
