Introduction
Autism was defined as a neuropsychiatric disorder characterised according to International Classification of Psychiatric Disorders (ICD-10) [World Health Organisation (WHO) 1992] and DSM-IV [American Psychiatric Association (APA) 1994] by a triad of impairments; impairment in social interaction and communication along with restrictive repetitive patterns of behaviour and activities. To diagnose autism by these classification systems, impairment in these three domains was required. The recently published DSM-5 (APA, 2013) introduces the new category of autism spectrum disorder (ASD), represented by impairment in two dimensions (1) social communication/social interaction and (2) restricted, repetitive behaviours and interests.
Kanner, first to use the term autism, described a syndrome that he called ‘early infantile autism’ (Kanner, Reference Kanner1943; Kanner & Eisenberg, Reference Kanner and Eisenberg1956). Autism was introduced as a childhood diagnosis in the ICD-8 in 1967 (WHO, 1967). The ICD-10 (WHO, 1992) and the DSM-IV (APA, 1994) had almost identical criteria for diagnosis of autism (ICD-10) and autistic disorder (DSM-IV).
Since Kanner originally described early infantile autism (Kanner, Reference Kanner1943) there has been considerable debate on the dimensional nature of the phenotype of ASDs. Many researchers agree that typical autism is part of a wider continuum of conditions that include Kanner’s early infantile autism and Asperger’s syndrome (Wing, Reference Wing, Schopler and Mesibov1988; Gillberg & Gillberg, Reference Gillberg and Gillberg1989). Manifestations of ASD in clinical practice vary in type and severity and can present with a variety of combinations of impairments (Wing, Reference Wing, Schopler and Mesibov1988). The recognition that autism can present with a range of symptoms on a continuum has led to a general trend of broadening of the criteria for diagnosing autism. These issues highlight the need to use well-studied instruments in the assessment of persons where there is a concern regarding diagnosis of ASD. It is important therefore to carefully examine the psychometric properties of the diagnostic instruments currently in use.
Measures used in the diagnosis of ASD
Different measures are used in the diagnosis of ASDs. Some of the measures are based on parent report such as the autism diagnostic interview-revised, (ADI-R, Lord et al. Reference Lord, Rutter and Le Couteur1994), the diagnostic interview for social and communication disorders (DISCO; Wing et al. 2002; Wing, Reference Wing2006), the developmental, dimensional and diagnostic interview (3DI; Skuse et al. Reference Skuse, Warrington, Bishop, Chowdhury, Lau, Mandy and Place2004), the social responsiveness scale (Constantino & Gruber, Reference Constantino and Gruber2005) and the social communication questionnaire (Rutter et al. Reference Rutter, Bailey and Lord2003). Other measures, for example, the autism diagnostic observation schedule (ADOS) (Lord et al. Reference Lord, Risi, Lambrecht, Cook, Leventhal, DiLavore, Pickles and Rutter2000) and the childhood autism rating scale (CARS; Schopler et al. Reference Schopler, Reichler, DeVellis and Daly1980) involve observation of the child. The content of the ADI-R closely mirrors the categorical descriptions of autism found in DSM-IV and ICD-10. The dimensional concept of autism is reflected in the DISCO (Wing & Gould, Reference Wing and Gould1979; Wing et al. 2002) and the 3DI (Skuse et al. Reference Skuse, Warrington, Bishop, Chowdhury, Lau, Mandy and Place2004).
The ADI-R is well established as a diagnostic instrument with good validity and reliability (Le Couteur et al. Reference Le Couteur, Rutter, Lord, Rios, Robertson, Holdgrafer and McLennan1989; Tadevosyan-Leyfer et al. Reference Tadevosyan-Leyfer, Dowd, Mankoski, Winklosky, Putman, McGrath, Tager-Flusberg and Folstein2003; De Bildt et al. Reference De Bildt, Sytema, Ketelaars, Kraijer, Volkmar and Minderaa2004; Lecavalier et al. Reference Lecavalier, Aman, Scahill, McDougle, McCracken, Vitiello, Tierney, Arnold, Ghuman, Loftin, Cronin, Koenig, Posey, Martin, Hollway, Lee and Kau2006) and is probably the most widely used instrument in autism research. Most of the earlier studies examining the variability of the phenotype were carried out using the ADI-R and the ADOS, which have been widely accepted as being the gold standard for clinical and research purposes. Constantino et al. (Reference Constantino, Davis, Todd, Schindler, Gross, Brophy, Metzger, Shoushtari, Splinter and Reich2003) and Lecavalier et al. (Reference Lecavalier, Aman, Scahill, McDougle, McCracken, Vitiello, Tierney, Arnold, Ghuman, Loftin, Cronin, Koenig, Posey, Martin, Hollway, Lee and Kau2006) have used the ADI-R in their validity studies, and Constantino et al. (Reference Constantino, Gruber, Davis, Hayes, Passanante and Przybeck2004), Szatmari et al. (Reference Szatmari, Mérette, Bryson, Thivierge, Roy, Cayer and Maziade2002) and Tadevosyan-Leyfer et al. (Reference Tadevosyan-Leyfer, Dowd, Mankoski, Winklosky, Putman, McGrath, Tager-Flusberg and Folstein2003) have examined the dimensions of autism using the ADI-R. A study by Ventola et al. (Reference Ventola, Kleinman, Pandey, Barton, Allen, Green, Robins and Fein2006) found agreement between ADOS-G, CARS and clinical judgement in diagnosis of toddlers but not the ADI-R.
Evidence from previous studies showed that social and communication impairment could not be separated into two different categories as described in ICD-10 and DSM-IV (Kamp-Becker et al. Reference Kamp-Becker, Ghahreman, Smidt and Remschmidt2009) and should be considered a single criterion (Lord et al. 1994, 2000; Tanguay et al. Reference Tanguay, Robertson and Derrick1998; Robertson et al. Reference Robertson, Tanguay, L’Ecuyer, Sims and Waltrip1999). The DSM-5 and the proposed ICD-11 recognise this difficulty and contain only two domains: (1) social communication/social interaction and (2) restricted, repetitive behaviours and interests (Rutter, Reference Rutter2011; Lord & Jones, Reference Lord and Jones2012; WHO, 2012; APA, 2013). These changes reflect substantial literature, which has shown that, in many cases, whether a particular behaviour is classified under social or communication domains is arbitrary (Gotham et al. Reference Gotham, Pickles and Lord2009). For example, non-verbal behaviours such as eye contact, gesture and facial expressions (grouped under social impairment in DSM IV (1994) and ICD-10 (1992) are social and communicative; conversation is a social use of language. These are now included within social communication/social interaction as are social aspects of imaginative play rather than in a separate domain of communication.
Studies that examined the proposed changes to DSM-5 report either good sensitivity (Huerta et al. Reference Huerta, Bishop, Duncan, Hus and Lord2012), or specificity, with (Frazier et al. Reference Frazier, Youngstrom, Speer, Embacher, Law, Constantino, Findling, Hardan and Eng2012) reporting both within the same sample.
Frazier et al. (Reference Frazier, Youngstrom, Speer, Embacher, Law, Constantino, Findling, Hardan and Eng2012) in their study evaluating the validity of the proposed DSM-5 criteria for ASD found that DSM-5 had superior specificity compared to DSM-IV-TR (0.97 versus 0.86). The sensitivity, however, was lower (0.81 versus 0.95) but still significant.
Matson et al. (Reference Matson, Belva, Horovitz, Kozlowski and Bamburg2012); looked at 227 individuals with intellectual disability divided into three groups: those meeting DSM-IV-TR criteria for ASD, those meeting ASD criteria according to proposed DSM-5 and controls with ID not meeting ASD criteria by either system. Results showed that individuals in the DSM-5 group showed significantly greater overall ASD symptoms than those in the other two groups.
McPartland et al. (Reference McPartland, Reichow and Volkmar2012) examined the sensitivity and specificity of the proposed DSM-5 diagnostic criteria for ASD and found that overall specificity was high with 94.9% of individuals accurately excluded from the spectrum. However, they found that sensitivity varied according to diagnostic subgroups (autistic disorder = 0.76; Asperger’s syndrome = 0.25; Pervasive Developmental Disorder - Not Otherwise Specified = 0.46). Sensitivity also varied according to cognitive ability. This was further supported by Sung et al. (Reference Sung, Goh, Tan, Chan and Liew2018) who found that when the DSM-5 (APA, 2013) and the DSM-IV-TR (APA, 1994) were applied against best estimate clinical diagnosis the DSM-IV-TR yielded slightly better sensitivity and specificity than the DSM-5.
The DISCO (Wing, Reference Wing, Schopler and Mesibov1988; Leekam et al. Reference Leekam, Libby, Wing, Gould and Taylor2002) is a semi-structured interview that enables algorithm diagnoses according to DSM and ICD. The DISCO is a diagnostic tool based on the concept of a spectrum of autistic disorders. This concept predated the earliest ICD and DSM criteria for autism (Wing & Gould, Reference Wing and Gould1979; Wing, Reference Wing, Schopler and Mesibov1988) and is therefore not constrained by existing international classifications. Wing et al. (Reference Wing, Leekam, Libby, Gould and Larcombe2002) provide a detailed description of the development of the DISCO. Unlike the ADI which was developed as a diagnostic tool, the DISCO was designed to systematically obtain information needed to compile an individual’s developmental history from birth and to describe their current pattern of behaviour, with the DISCO covering a wider range of developmental domains. The DISCO contains a very large number of items covering specific examples of behaviour seen in autism, from the common to the rare. Although the primary purpose of the DISCO was not to provide a diagnostic category, this is often necessary for an individual to gain clarity about their condition and to gain access to services. The diagnostic algorithms for DISCO-9 were written for this purpose. These algorithms included three of the sets of standard international diagnostic criteria for pervasive developmental disorders and their sub-groups; DSM-111-R (APA, 1987), ICD-10 (WHO, 1992, 1993) and DSM-IV (APA, 1994). An advantage of the DISCO is that because it collects a wide range of factual information without bias toward any diagnostic system, the items within the DISCO did not need to be modified when changes are made to DSM and ICD criteria. On the other hand, because the criteria for the international systems need to be fitted to the DISCO items rather than vice versa, provisional algorithms that select items from the DISCO also need to be fine-tuned to accurately connect diagnostic criteria to DISCO items. The original version of the DISCO has been revised a number of times – DISCO-9; DISCO-10 and the current version DISCO-11 (2007) which was used in this study. From this point in this paper, the term used will be DISCO rather than DISCO-11.
Evidence base for the use of DISCO in diagnosis of ASD
A number of studies have supported the use of the DISCO for diagnosis of ASDs according to DSM-IV and ICD-10. These have noted positive inter-rater reliability for algorithm items (Leekam et al. Reference Leekam, Libby, Wing, Gould and Taylor2002; Wing et al. Reference Wing, Leekam, Libby, Gould and Larcombe2002; Nygren et al. Reference Nygren, Hagberg, Billstedt, Skoglund, Gillberg and Johansson2009), good agreement between clinical diagnosis and the DISCO algorithms for ICD-10 (Leekam et al. Reference Leekam, Libby, Wing, Gould and Taylor2002), good agreement between the DISCO and ADI-R (Nygren et al. Reference Nygren, Hagberg, Billstedt, Skoglund, Gillberg and Johansson2009) and high sensitivity and specificity for DISCO-11 classifications in differentiating ASD and non-ASD (Maljaars et al. Reference Maljaars, Noens, Scholte and van Berckelaer-Onnes2011). Maljaars et al. (Reference Maljaars, Noens, Scholte and van Berckelaer-Onnes2011) in their study of young and low functioning children also found substantial agreement between DISCO and ADOS classifications and found that the correlations between raw scores of the DISCO and ADOS algorithms were high. Excellent criterion validity was found by Nygren et al. (Reference Nygren, Hagberg, Billstedt, Skoglund, Gillberg and Johansson2009) when the DISCO was compared with clinical diagnosis and an investigator based diagnostic interview. A recent study using the DISCO demonstrated that good levels of both sensitivity and specificity can be achieved for a diagnostic algorithm adhering to DSM-5 criteria that is suitable across age and ability level (Kent et al. Reference Kent, Carrington, Le Couteur, Gould, Wing, Maljaars, Noens, van Berckelaer-Onnes and Leekam2013). This paper also found excellent overall internal consistency (Cronbach’s alpha = 0.95).
A study by Carrington et al. (Reference Carrington, Kent, Maljaars, LeCouteur, Gould, Wing, Noens, Van Berckelaer-Onnes and Leekam2014), based on analysis of the DISCO DSM-5 algorithm aimed to ‘investigate DSM-5 diagnostic criteria to identify those behaviours that best discriminated between individuals with ASD and non-ASD clinical diagnoses and which could therefore be considered “essential” for the diagnosis of DSM-5 ASD’. They highlight the importance of disentangling symptoms that are common across neurodevelopmental disorders from those most specifically identified with ASD. They considered that this could lead to more streamlined and efficient diagnostic practices. Carrington et al identified highly discriminating ‘essential’ behaviours. A reduced item set of 48 items had good predictive ability but lacked sensitivity for higher functioning participants. An adjusted item set of 54 items had good sensitivity when applied to higher ability participants and performance was comparable to the published full DISCO DSM-5 algorithm. Their results highlighted that social-communication behaviours highly discriminate between ASD and other, non-ASD clinical diagnoses. The paper reports further that items measuring sensory behaviours were among the most highly discriminating items in the restricted and repetitive behaviour domains.
Rationale for studying the DISCO
This study looks at the usefulness of the DISCO as a tool to aid in the diagnosis of ASD according to DSM-5 and examines the initial psychometric properties of the tool. The DISCO DSM-5 algorithm includes multiple items for many of the behaviours described in DSM-5. The inclusion of these different examples of behaviours means that individual variations in symptom profile, for example, the different types of sensory sensitivity can be captured within the DSM-5 algorithm. The DISCO and its factor structure has not been as extensively studied as other assessment tools.
The study was undertaken in a clinical sample of children in a tertiary children’s hospital in Ireland. The study was developed by the specialist neurodevelopmental team in the hospital in response to concerns by the team regarding the increasing number of children being diagnosed by the team with ASD. As the specialist team were using the DISCO routinely as part of their assessment, they wished to check its validity given that it was a less frequently used instrument. The psychiatrist and the psychologist on the team had undergone training in the administration of the DISCO when the team was first established. Prior to undertaking the research, the research registrar also received this training. The training was administered by the Centre for Social and Communication Disorders (UK). It involved 5 days training initially with a multidisciplinary group of trainees learning the underlying concepts and practising the administration of the tool. There was a follow-up training day 3 months later. Trainees, having administered the DISCO and presenting same to trainers were evaluated and subject to that evaluation were licenced to administer the DISCO. Follow-up consultation was available if needed and further training provided when changes were made to the DISCO manual (e.g. revision from DISCO 10 to DISCO 11). All of the clinical and research team received training on the administration of the ADOS.
The DISCO was chosen by the team as an assessment tool because it was designed to cover the broad autism spectrum, at all ages and different levels of development. The DISCO covers a wide range of developmental domains, including aspects of self-care, independence, visuo-spatial and other developmental domains. It provides a profile of an individual’s strengths, needs and difficulties, including sensory symptoms and possible coexisting conditions such as motor coordination difficulties and attention deficit/hyperactivity disorder. The DISCO also includes more detail on untypical responses to sensory stimuli, motor stereotypies, repetitive routines and emotional disturbance than other diagnostic tools. The DISCO’s ‘bottom up’ approach to gathering information in relation to the child patient, having been designed to obtain, systematically, information needed to compile an individual’s developmental history from birth and a description of their current pattern of behaviour, aids in assessing the participant’s needs and in devising and planning appropriate interventions. For these reasons, it is an instrument that suited the team’s work context.
Cox & Rutter (Reference Cox, Rutter, Rutter and Hersov1985) quoted experimental evidence to show that systematic questioning obtained fuller and more detailed information than free reporting of children’s symptoms. Work using contemporaneous videotapes (Osterling et al. Reference Osterling, Dawson and Munson2002) suggests that parents may not notice subtle abnormalities in the first year of life. This emphasises the importance of incorporating information from other sources in order to make an accurate diagnosis of ASDs. The DISCO fulfils these criteria more fully than many other research instruments, designed, as it was to provide a comprehensive description of individual characteristics and needs.
Aim of the present study
The aim of the present study is to examine the psychometric properties of the DISCO. Specific research questions examined were:
-
• Does the factor structure of the DISCO reflect the dyad of impairment in ASD?
-
• Is there evidence of diagnostic stability over time using the DISCO?
Method
Participants
In total, 85 children, who were diagnosed as having ASD over a 3-year period in a tertiary referral service for children and young people with complex developmental disorders, were invited to take part in this study. Of those, 65 participants agreed to be involved in the study. All the children in the study were seen initially (Time 1) when they were aged less than 7 years for the assessment of complex developmental difficulties by a specialist multidisciplinary mental health liaison team including a consultant child & adolescent psychiatrist, a clinical psychologist, a speech and language therapist, an occupational therapist and a social worker. The children were referred to the team with developmental queries including ASD, by consultants in the hospital, mainly by developmental paediatricians, neurologists, neonatologists, general paediatricians and ENT surgeons. As the team worked in a national paediatric hospital, children could be referred from the whole country (population 5 million approximately), but in practice children were mainly assessed from the Dublin area (1.5 million approximately) and appropriate local services were identified to the referrer for children who lived further afield. However, given the tertiary nature of the hospital children with complex presentations were seen for assessment or a second opinion by the team.
All children in the study received a diagnosis of autism or ASD (Time 1) by consultant psychiatrists experienced in diagnosing children with ASD. Of the 65 children, 36 (55%) met the ICD-10 criteria for a diagnosis of autism/ASD on the basis of clinical assessment alone. The other 29 (45%) met the criteria for an ICD-10 diagnosis autism/ASD (autism (n = 25); atypical autism (n = 2); Asperger’s syndrome (=1); non-ASD (n = 1) by the use of DISCO and ADOS (standardised instruments that were introduced to the clinic during that time) in addition to clinical assessment. The child diagnosed as non-ASD met the criteria for ASD on ADOS assessment but not on the DISCO. Of the 65 children in the study, 50 had cognitive assessments at T1.
At Time 1 the 65 participants, 51 males (78%) and 14 females (22%), were aged 2 years 6 months – 6 years 4 months (30–76 months; mean age = 3 years 10 months; SD = 10 months). These participants were reassessed 2 to 5 years later (Time 2) using the DISCO-11 and the ADOS. They had a mean age of 7 years 9 months (SD = 23 months) with a range of 4 years 9 months to 13 years 2 months (57–158 months).
The DISCO was administered by a licensed clinician (research registrar in child and adolescent psychiatry). The child’s parent(s) or carer(s) was interviewed either in the hospital clinic or in their home. The participants assessed by the ADOS received a consensus diagnosis by members of the research team (research registrar in child and adolescent psychiatry, research psychologist and research speech and language therapist).
Instruments
The Diagnostic Instrument for Social and Communication disorders (DISCO-11)
The DISCO is a semi-structured interview developed for use in the diagnosis of autism. Originally, the DISCO was designed for clinical use to provide a comprehensive description of individual characteristics and needs, but the interview can be used for classification in both clinical practice and for research. For this purpose, computerised algorithms have been developed. These algorithms provide classifications based on official classification systems such as DSM-5 (APA, 2013) ICD-10 (WHO, 1992) and DSM-IV (APA, 1994). There are also algorithms for early infantile autism (Kanner & Eisenberg, Reference Kanner and Eisenberg1956), autistic spectrum disorder (Wing & Gould, Reference Wing and Gould1979) and Asperger’s syndrome according to Gillberg’s criteria (Gillberg & Gillberg, Reference Gillberg and Gillberg1989; Gillberg, Reference Gillberg and Frith1991; Leekam et al. Reference Leekam, Libby, Wing, Gould and Gillberg2000). The DISCO is designed to diagnose autism by allocating weighted values to particular questions (items) asked of informant(s). Algorithms are devised to reach a diagnosis. The algorithms for DSM-5 based on a selection of 80 DISCO items were used in this study, excluding criterion C and criterion D (as outlined below) as they were found not to be a significant contributor to the diagnosis of ASD as per DSM-5, Carrington et al. (Reference Carrington, Kent, Maljaars, LeCouteur, Gould, Wing, Noens, Van Berckelaer-Onnes and Leekam2014). Furthermore, criteria C and D are not algorithm items.
Criterion C-symptoms must be present in the early developmental period (but may not become fully manifested until social demands exceed limited capacities or may be masked by learned strategies in later life).
Criterion D-symptoms cause clinically significant impairment in social, occupational or other important areas of functioning.
The DISCO was used in this project to gather information from parents in relation to the participants assessed. The interview takes 2 to 3 hours to administer and consists of more than 300 questions under different sections: ‘Family Medical Background and Identifying Information’; ‘Infancy’; ‘Developmental Skills’; ‘Repetitive Stereotyped Activities’; ‘Emotions’; ‘Maladaptive Behaviour’; ‘Interviewer’s Judgement of Quality (of social interaction, social communication, imagination, activities)’; ‘Psychiatric Disorder & Forensic Problems’. (Given the age of the children assessed the section ‘Psychiatric Disorder & Forensic Problems’ were not used in this project.) The questions are divided into three groups pertaining to current level of function, delay in acquiring relevant skills and untypical behaviours associated with the relevant skills. Most of the questions are scored on three levels of severity: ‘marked problem’, ‘minor problem’ or ‘no problem’. The DISCO distinguishes between ‘ever’ and ‘current’ ratings of the child’s behaviour. The ‘ever’ rating is retrospective and the ‘ever’ includes ‘current’. In this way, a distinction is made between current symptoms and behaviour that has ever been present.
The 80 individual items were recoded into 0 (impairment not present) and 1 (impairment present). The 80 items were subdivided into 7 items, (See Table 1), with the number of impairments present counted for each of the items. This gave each participant scores on the 7 items, with higher scores indicating more evidence of difficulty in the area.
The 7 items of the DISCO-11 based on the DSM-5
Statistical analyses
The 7 items of the DISCO-11 were subjected to factor analysis using principal component analysis (PCA). The aim of this analysis was to examine if the DISCO showed the two-factor structure discussed above in the present sample. As this research was an exploratory analysis, PCA was deemed more appropriate than confirmatory factor analysis. Varimax rotation was used based on the assumption that the structures described above reflect different aspects of autism that may not be related (as an oblique rotation would assume). The factors identified were further examined for their internal consistency using Cronbach’s alpha. Finally, the stability of diagnosis on the DISCO was examined for a subset of 29 participants, for whom the DISCO was conducted at both Time 1 and Time 2 (which was between 2 and 5 years later). As the DICSO allowed for the identification of a diagnosis at each time point, the McNemar test was used to assess if the dichotomous variable of having a diagnosis (Yes/No) was significantly different for participants from Time 1 to Time 2.
Results
Factor structure & reliability
An initial exploratory factor analysis (n = 65) was conducted using principal component analysis with Varimax rotation and Kaiser Normalisation. This revealed the presence of two factors with Eigen values exceeding 1, with a Scree Plot confirming these two factors with a break after the second component. The rotated solution explained 45.65% (repetitive and restricted interests), 19.62% (social communication), of variability respectively. These two components explained a total of 68% of variance (Table 2).
When these factors were examined for their reliability, the Cronbach’s alpha for the overall DISCO score was good at 0.779. Following this, the reliability of each factor was examined separately. The restricted and repetitive patterns of behaviour, interests or activities subscale showed good reliability (Cronbach’s Alpha = 0.735), as did social communication and social Interaction subscale (Cronbach’s Alpha = 0.770).
Based on the factor analysis and the reliability check, the 7 items were combined into a single variable representing the overall level of ASD symptom severity assessed using the DISCO, with higher sores indicating greater severity (based on the response set of ‘no problem, ‘minor problem’ and ‘marked problem’ described in the methods). The scores for each of the subscales were also calculated, with higher scores indictive of greater symptom severity. Descriptive statistics for the scale and subscales are reported in Table 3.
Two factor solution to exploratory factor analysis
Descriptive statistics for average severity of symptoms based on items on the overall scale and the two subscales for the total sample (n = 65) on DISCO-11 items
Stability of diagnosis of ASD
Finally, a McNemar’s test was used to examine the stability of the diagnosis of ASD using the DISCO over time. A diagnosis of ASD or non-ASD was coded based on the DISCO at both Time 1 and Time 2 for the 29 participants who had a DISCO completed at both time points. The McNemar test showed no significant difference in the coded response for participants at Time 1 to Time 2, suggesting there was no statistically significant difference in the presence/absence of a diagnosis for the group, suggesting evidence of stability in the diagnosis over time (p = 0.125).
Discussion
The aim of this study was to examine the psychometric properties of the DISCO-11.
Two research questions in particular were examined as follows:
-
Does the factor structure of the DISCO reflect the dyad of impairment in ASD? And
-
Is there evidence of diagnostic stability over time using the DISCO?
The results revealed two factors resembling the DSM-5 algorithms, with almost 68% of the variance in the data accounted for. Each of the two subscales (which resulted from the factor analysis) showed acceptable levels of reliability, as did the overall DISCO score. The results showed high test–retest reliability of DISCO-11 with no significant difference in diagnosis from Time 1 to Time 2.
Validity of DISCO
In the factor analysis of the DISCO-11, 7 items were derived using the DISCO-11 DSM-5 algorithms. These 7 items were the original sub categories of the DISCO-11 representing the different domains of impairment. The two factors explored based on DSM-5 criteria were social communication difficulties and restricted and repetitive interests. The study revealed that the items that loaded strongly on the two factors were the original algorithms of the DISCO-11 DSM-5, which comprise the deficit domains of ASD.
In our internal reliability analysis of the resultant factors of the DISCO-11, good internal consistency was observed between the resultant factors with a Cronbach’s alpha coefficient of 0.78. When the social communication factors and the repetitive and restricted interests’ factors were analysed separately, the Cronbach’s alpha coefficient for the social communication subscale (Cronbach’s Alpha = 0.77) showed good reliability, as did the repetitive and restricted interests subscale (Cronbach’s alpha = 0.74).
The results highlight items essential for the diagnosis of DSM-5 ASD, social communication behaviours and restricted repetitive behaviours. As the DISCO includes more detail on untypical responses to sensory stimuli, and repetitive routines than other diagnostic tools it provides a wealth of information relevant to DSM-5 ASD diagnosis, as well as providing a detailed developmental profile of the participant. Carrington et al. (Reference Carrington, Kent, Maljaars, LeCouteur, Gould, Wing, Noens, Van Berckelaer-Onnes and Leekam2014) showed that items measuring sensory behaviours were among the most highly discriminating items in the domain of the restricted repetitive behaviour. This was not the case in our paper although the number of items in this area was significant. However outcome on the DSM-5 algorithms is not dependent only on the number of items on which an individual scores; the algorithm rules specify a particular pattern of symptoms, namely symptoms in all three of the social-communication sub-domains and at least two of the four in the restricted repetitive sub-domains. The results of our paper also show this pattern.
The stability of diagnosis between T1 and T2 in our study is in contrast to the findings of others (Tanguay Reference Tanguay2011; McPartland et al. Reference McPartland, Reichow and Volkmar2012; Kulage et al. Reference Kulage, Smaddone and Cohn2014) who found that a significant number of people clinically diagnosed with ASD did not meet criteria for ASD according to DSM-5. The DISCO was not the measurement tool used in these papers. Our paper showed good stability over time.
While this study was done in a specialist paediatric service the profile of patients referred to the service are similar to that referred to a community service with the exception of children with very complex medical and neurodevelopmental needs. The team found that the detailed assessment (DISCO, ADOS, cognitive, speech and language and occupational therapy assessments) of the children referred, generally completed over a 6-week period, helped to bring parents on board with the diagnosis. Initially many, especially first-time parents, were unsure if their child had a significant problem, often believing that speech was the only issue. Helped by the thoroughness of the assessment and the support from team members, especially the social worker, the parents generally, although shocked and grief stricken were enabled to accept the situation and empowered to move to the next phase in helping their child. This, while a resource heavy process, was a therapeutic way of providing ASD assessments and in the long run thought to be cost effective in terms of minimising denial, ongoing searches for other answers and being enabled to accept appropriate intervention. This was not, however, evaluated. This type of service should be applicable to non-specialist child and adolescent mental health services teams, but because of the severely limited resources of such teams and the demands on those teams from the adolescent cohort, the assessments of children with neurodevelopmental problems are not prioritised. While the DISCO as an assessment tool, as outlined earlier, provides a very detailed developmental profile of the child patient its administration is time consuming (2–3 hours). Carrington and colleagues’ (Reference Carrington, Kent, Maljaars, LeCouteur, Gould, Wing, Noens, Van Berckelaer-Onnes and Leekam2014) study, as outlined earlier, aimed to identify a set of ‘essential’ behaviours sufficient for the diagnosis of DSM-5 ASD. The good psychometric properties of the reported item sets in the Carrington paper suggests that the search for items essential for the diagnosis of DSM-5 ASD may have identified items that are potentially useful for clinicians and researchers in the development of efficient and focussed ASD diagnostic processes. Clearly it would be advantageous to identify the most salient clinical items for DSM-5 as this process could guide the development of more efficient clinical approaches to diagnosis. Although this and other papers show the psychometric properties of the DISCO as being robust, further research is needed to explore these properties.
Strengths and limitations
This study adds to the growing body of research examining the psychometric properties of the DISCO, in particular, its factor structure. Our results reflect the importance of using a combination of all the impairment domains together, which make up the DISCO-11 for the assessment and diagnosis of ASDs and support that the DISCO-11 is a valid instrument for the diagnosis of ASDs using DSM-5 algorithms.
This study was carried out in a specialist early assessment service that may not generalise to the rest of the population and is limited by the relatively small number of participants. This is an area where further research is needed.
This study is a good example of ecological validity investigating, as it does, the current assessment procedures and presents the opportunity to evaluate these assessment procedures.
A limitation of the existing literature relating to DSM-5 is that data was typically collected according to DSM-IV-TR criteria and mapped on to DSM-5 descriptions. (e.g. Taheri & Perry, 2012). In this study, the DISCO-11 was used to collect data for assessment and diagnostic purposes, the latter according to ICD-10, and then mapped on to DSM-5 descriptions also.
The interviewer was aware that all cases had previously been diagnosed with autism by clinical judgement plus or minus DISCO and ADOS before the DISCO interview. Furthermore as all children had previously been diagnosed with autism parents/carers’ answers might have been influenced by familiarity with the types of questions asked.
Further research is required to confirm our findings using the DISCO with a larger sample and to demonstrate the association between the factor structure of the DISCO with newer assessment tools including the ADOS-2 (Lord & Jones, 2012), and the proposed ICD-11 structure.
The DISCO-11 has not been as extensively studied as the ADI-R, but it seems to have good psychometric properties and can be considered in future with other instruments like the ADOS for both clinical and research purposes.
Conclusion
This paper was set out to examine the psychometric properties of the DISCO. Our results revealed two factors resembling the DSM-5 algorithms, as used in DISCO 11, which were named as social-communication, and restricted and repetitive behaviours. Acceptable internal reliability was found for the overall DISCO score and the subscales of social impairment and repetitive activities. There was also evidence of stability of diagnosis based on test–retest over time. This study supports that the DISCO-11 shows potential as a valid and reliable instrument that can be used both for clinical and research purposes.
Acknowledgements
We wish to acknowledge Professor Louise Gallagher, Linn Dara Child and Adolescent Mental Health Services, St. James’s Hospital Dublin Ireland. We also acknowledge the help of children, their parents, and the team who participated in the research project.
Conflicts of interest
Authors have no conflicts of interest to disclose.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committee on human experimentation with the Helsinki Declaration of 1975, as revised in 2008. Ethical approval from the hospital’s ethics committee was obtained and assessments were carried out following informed consent from parents and from professionals from external agencies involved with the child participant.
Financial support
The research was supported by a grant from Temple Street Children’s University Hospital Fundraising Department, Temple Street, Dublin Ireland.
