Introduction
The DSM-5 Obsessive-Compulsive Spectrum Sub-Work Group of the Anxiety, Obsessive-Compulsive Spectrum, Posttraumatic, and Dissociative Disorders Work Group is currently recommending the creation of a new diagnostic category in the forthcoming edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) named Hoarding Disorder (HD) (APA, 2010; Mataix-Cols et al. Reference Mataix-Cols, Frost, Pertusa, Clark, Saxena, Leckman, Stein, Matsunaga and Wilhelm2010). The provisional diagnostic criteria for HD (Table 1) are based on pre-existing criteria developed by Frost & Hartl (Reference Frost and Hartl1996), which had already been widely adopted by the field in various forms.
Provisional diagnostic criteria for Hoarding Disorder
* The Work Group is considering alternative wording: ‘Persistent difficulty discarding or parting with possessions, regardless of their actual value’.
The above proposed criteria for HD seem to have adequate clinical face validity, as they are based on hundreds of cases from around the world that have been fairly consistently described. The criteria are already being routinely used in research and clinical settings by a number of groups in the United States, UK, Australia, Italy, Germany and Japan. However, the sensitivity, specificity and acceptability of the recently proposed diagnostic criteria have yet to be formally tested (Mataix-Cols et al. Reference Mataix-Cols, Frost, Pertusa, Clark, Saxena, Leckman, Stein, Matsunaga and Wilhelm2010). Using a clinical vignette methodology, the goals of this study were two-fold: (1) to assess the sensitivity and specificity of the proposed diagnostic criteria for HD in an international sample of obsessive-compulsive disorder (OCD)/hoarding experts as well as mental health professionals who are not necessarily experts in OCD/hoarding; (2) to examine the perceived usefulness and acceptability of the criteria in the above groups of professionals.
We hypothesized that the proposed criteria for HD would show good sensitivity and specificity across a wide range of professionals (e.g. experts versus non-experts, psychiatrists versus other mental health professionals, senior versus junior faculty, etc.). We also hypothesized that the new diagnosis would be deemed acceptable for both professionals and sufferers alike.
Methods
Hoarding survey
The study was reviewed and approved by the DSM-5 Scientific Review Committee. The American Psychiatric Institute for Research and Education Institutional Review Board (IRB) determined that the survey was exempt from IRB review.
An online survey was developed (see Supplementary online material 1) and hosted in www.surveygizmo.com. After an initial welcome screen, responders were presented with a clinical vignette adapted from one of the case descriptions of the DSM-IV Casebook (Spitzer et al. Reference Spitzer, Gibbon, Skodol, Williams and First2002) entitled ‘Something of Value’ (pp. 15–17). Participants were instructed to use their clinical judgement to choose the most appropriate diagnosis from a list of options, strictly adhering to DSM-IV-TR criteria. The available options were: (1) anxiety disorder not otherwise specified; (2) dementia/organic brain disorder; (3) OCD; (4) obsessive-compulsive personality disorder; (5) personality disorder not otherwise specified; (6) schizophrenia; (7) other mental disorder; (8) ‘I don't think this behaviour is pathological enough to be considered a disorder’; (9) ‘I think this behaviour is pathological and should be considered a disorder but it does not fit into any of the current DSM-IV-TR categories’. The purpose of this initial vignette was to explore to what extent the current DSM-IV-TR categories are suitable for cases of HD.
The following screens contained information about the phenomenology of hoarding behaviour in different mental disorders (with special emphasis on the differential diagnosis between HD and OCD) as well as the provisional diagnostic criteria for HD (APA, 2010).
Responders were then presented with eight clinical vignettes (approximately 200 words per vignette) of individuals who endorsed some kind of hoarding behaviour. Participants were asked to choose one or two diagnoses among the following options: (1) dementia/organic brain disorder; (2) HD; (3) OCD; (4) obsessive-compulsive personality disorder; (5) schizophrenia; (6) other mental disorder; (7) ‘I don't think this patient warrants any diagnosis’. At the end of each vignette, participants were also asked to select which of the individual criteria for HD were endorsed in each case. Given that all cases displayed some form of hoarding behaviour, the first two criteria for HD (‘Persistent difficulty discarding or parting with possessions’ and ‘Strong urges to save items and/or distress associated with discarding’) were met in all eight cases.
Some cases were adaptations of hoarding cases described in the literature. Case no. 1 is an adaptation of one of the cases described by Anderson et al. (Reference Anderson, Damasio and Damasio2005), vignette no. 3 was adapted from Pertusa et al. (Reference Pertusa, Fullana, Singh, Alonso, Menchon and Mataix-Cols2008) and vignette no. 8 from Plimpton et al. (Reference Plimpton, Frost, Abbey and Dorer2009), while the rest (nos. 2, 4, 5, 6 and 7) consisted of fictional cases written by the authors. All cases were carefully chosen to meet criteria for the intended diagnoses and to appear realistic. Several cases were deliberately chosen to force the clinician to make a differential diagnoses and to separate normal from abnormal hoarding behaviour. Four cases (nos. 3, 6, 7 and 8) met criteria for HD. Cases no. 3 and no. 6 were individuals with prototypical HD. Case no. 7 was a 37-year-old woman with both OCD and co-morbid HD; case no. 8 was a 10-year-old girl with HD. In three cases, hoarding behaviour was secondary to other mental health disorders (dementia in case no. 1, OCD in case no. 2 and schizophrenia in case no. 5). Finally, case no. 4 was an individual with subclinical hoarding behaviour who did not meet criteria for HD or other psychiatric diagnoses.
After completing the survey, participants were asked to provide background information: name and email (optional); country; profession (psychiatrist, psychologist, other); type of professional activity (clinician, researcher or both); level of seniority (senior faculty or resident/trainee/student); whether OCD and related disorders were their main area of interest or expertise; whether they considered themselves hoarding experts; whether they had seen patients fitting the diagnostic criteria for HD; whether they were familiar with DSM-IV-TR and DSM-5 diagnostic criteria. The survey also contained two questions about the acceptability of the proposed HD criteria for both professionals and sufferers (very acceptable, somewhat acceptable, not too acceptable, not at all acceptable). Participants were also asked whether HD should be included in the DSM-5 Main Manual or in the Appendix of Criteria Sets Provided for Further Study. Finally, responders were given the option to write their views on the proposed HD specifiers (excessive acquisition and degree of insight) and any other comments (open field).
The survey was first piloted among a small sample of colleagues who are experts in the fields of OCD and hoarding to ensure that there were no programming errors.
Participants and procedures
The survey was administered to two different samples. The first sample consisted of OCD and/or hoarding experts from all over the world. We defined ‘OCD or hoarding expert’ as: (a) someone who has published at least one indexed paper on OCD and/or hoarding between 1996 and March 2010 as corresponding author; and/or (b) an academic and/or clinician with a strong interest in OCD and/or hoarding whose professional activities involve research and/or clinical work in the fields of OCD and/or hoarding. The second sample comprised 500 practising psychiatrists randomly selected from the American Psychiatric Association's (APA) member directory.
The first sample (hereafter, ‘expert sample’) was contacted via email with a personalized cover letter explaining the aims of the study and a link to the online survey. They were given 1 month to complete the survey. To improve the response rate, a reminder was sent to those who had not yet completed the survey 10 days before the deadline. The email addresses were obtained from databases such as PubMed and Elsevier's ScienceDirect. The key words ‘hoarding’, ‘OCD’ and ‘obsessive-compulsive’ were used to track the corresponding authors of papers on OCD and/or hoarding that had been published between 1996 and March 2010. A total of 590 email addresses were retrieved using this procedure, of which 80 (14%) were invalid or were returned by the server. Therefore, a total of 510 valid emails were sent out. A text version of the survey was sent by email upon request to seven users (1%) who reported technical problems to access the online survey. Altogether, 155 completed surveys were returned from the initial 510 email addresses, representing a response rate of 30%. Participants were encouraged to widely distribute the survey among as many of their colleagues as possible, including clinicians, researchers, psychiatrists, psychologists or other mental health professionals. This method provided a further 56 completed surveys. Thus, a total of 211 completed surveys were received (see Fig. 1a).
(a) Expert sample and (b) American Psychiatric Association sample (randomly selected from the members' directory).
The second recruited sample (hereafter, ‘APA sample’) comprised 500 psychiatrists selected at random from the APA member directory. An email was sent with a cover letter explaining the aims of the study and a link to the online survey. Participants were asked to complete the survey within 6 weeks. To improve the response rate, participants who had not yet completed the survey were sent up to four reminders before the deadline. Given that all participants from this sample were contacted through the Division of Research of the APA, we did not have access to the names or email addresses of these participants. A total of 17 email addresses (3%) were invalid or were returned by the server. Thus, 483 valid emails were sent. Altogether, 48 completed surveys were returned, representing a response rate of 10% (see Fig. 1b).
Although both samples completed the same survey, the diagnostic criteria for HD underwent minor changes shortly before the survey was sent to the second sample (APA sample). These minor changes were made in response to the initial feedback received from both the professional community and the general public. A summary of the differences between both sets of criteria can be found in the Supplementary online material 2.
Statistical analyses
The completed surveys were downloaded into a spreadsheet and analysed using SPSS version 18 (SPSS Inc., USA) and Microsoft Excel 2007 (Microsoft Corporation, USA). Percentages for each question were calculated for both samples and between the different subgroups in the sample of OCD/hoarding experts (e.g. psychiatrists versus psychologists, experts versus non-experts). Sensitivity and specificity for each diagnosis as well as for each individual diagnostic criterion were also calculated. Sensitivity was defined as the proportion of clinical vignettes describing a case of HD that were appropriately diagnosed as such by the participants, whereas specificity was defined as the proportion of clinical vignettes describing a case without HD in which HD was not diagnosed.
Results
Characteristics of the participants
Characteristics of the two different samples are summarized in Table 2. The expert sample (n=211) was heterogeneous, consisting of professionals from around the world, albeit mainly from Europe (n=90, 43%) and North America (n=78, 37%). The majority of responders worked as clinicians and researchers (n=146, 69%). About half of the responders were psychiatrists (n=98, 46%), while the rest were mainly psychologists (n=87, 41%). In total, 26 responders (12%) were trainees, residents or students. The vast majority of the sample (n=172, 82%) considered OCD and related disorders as their main area of interest and about one-fourth of these considered themselves hoarding experts (n=48, 28%).
Characteristics of the expert and American Psychiatric Association (APA) samples
OCD, Obsessive-compulsive disorder.
Percentages are rounded to the nearest integer; in some cases totals may exceed 100%.
In contrast, the APA sample (n=48) was very homogeneous. All participants were American and worked as psychiatrists. About two-thirds (n=31, 65%) worked as clinicians and only three (6%) were trainees, residents or students. The great majority stated that OCD and related disorders were not their main area of interest (n=44; 92%). Only one participant (2%) considered him/herself a hoarding expert.
Classification of the DSM-IV Casebook vignette
As described above, participants were first asked to diagnose a case adapted from the DSM-IV Casebook (Spitzer et al. Reference Spitzer, Gibbon, Skodol, Williams and First2002) describing an individual with hoarding behaviour (case ‘Something of value’, pp. 15–17). This initial vignette provides information about the diagnosis that participants would consider more appropriate for an individual with pathological hoarding behaviour in the absence of other OCD symptoms when they strictly adhere to current DSM-IV-TR criteria. Of note, this case was diagnosed as ‘Anxiety Disorder Not Otherwise Specified’ or, alternatively, ‘Personality Disorder Not Otherwise Specified’ in the DSM-IV Casebook. The results are shown in Fig. 2.
Percentatge of responses to the initial vignette taken from the DSM-IV Casebook and originally diagnosed as ‘Anxiety Disorder Not Otherwise Specified’ (or, alternatively, ‘Personality Disorder Not Otherwise Specified’). APA, American Psychiatric Association.
‘Obsessive-Compulsive Disorder’ was the most frequent diagnosis in the initial vignette in both samples (expert: n=124, 59%; APA: n=19, 40%). The second most frequent response was ‘I think this behaviour is pathological and should be considered a disorder but it does not fit into any of the current DSM-IV-TR categories’ (expert: n=62, 29%; APA: n=14, 29%). The options ‘Anxiety Disorder Not Otherwise Specified’ or ‘Personality Disorder Not Otherwise Specified’ were chosen by two participants (1%) in the expert sample and seven participants (15%) in the APA sample.
Sensitivity and specificity of the proposed HD diagnosis
Sensitivity of the HD diagnosis was high (85%) in the expert sample and somewhat lower (74%) in the APA sample. Specificity was very high (>89%) in both samples (Table 3). Importantly, sensitivity and specificity did not change significantly according to the participants' professional background, type of professional activity, level of seniority or whether they had substantial experience with hoarding cases (Table 4).
Sensitivity and specificity of the Hoarding Disorder diagnosis in the expert and American Psychiatric Association (APA) samples
Percentages are rounded to the nearest integer.
Sensitivity and specificity (%) of the Hoarding Disorder diagnosis in the expert sample according to responder's background
OCD, Obsessive-compulsive disorder.
Percentages are rounded to the nearest integer.
Examination of individual cases revealed that the co-morbid OCD+HD case (no. 7) and the paediatric HD case (no. 8) had the lowest sensitivity and specificity. The remaining cases had a very high percentage of correct responses, ranging from 77 to 99% (Table 5).
Percentage of correct diagnoses among eight clinical cases comprising the survey
APA, American Psychiatric Association; OCD, obsessive-compulsive disorder; HD, Hoarding Disorder.
Percentages are rounded to the nearest integer.
Sensitivity and specificity of the individual diagnostic criteria for HD were acceptable in both samples (Table 6). Criteria C (‘The symptoms result in the accumulation of a large number of possessions that fill up and clutter active living areas’) and D (‘Symptoms cause clinically significant distress or impairment’) showed the highest sensitivity and specificity (>85%). Criterion F (‘Hoarding symptoms not restricted to the symptoms of another mental disorder’) showed the lowest sensitivity, whereas Criterion E (‘Hoarding symptoms not due to a general medical condition’) showed the lowest specificity.
Sensitivity and specificity of the individual diagnostic criteria for Hoarding Disorder in the expert and American Psychiatric Association (APA) samples
Percentages are rounded to the nearest integer.
Acceptability of the proposed HD to professionals and sufferers
Regarding the acceptability of the proposed diagnostic criteria for HD, the vast majority of participants (experts=94%; APA=90%) regarded them as very or somewhat acceptable (see Fig. 3a). Similarly, most participants (experts=97%; APA=90%) expected the proposed criteria to be very or somewhat acceptable for the sufferers themselves (see Fig. 3b).
Perceived acceptability of the proposed Hoarding Disorder criteria for (a) professionals and (b) patients. APA, American Psychiatric Association.
Placement in the DSM-5 manual
Most (71%) participants in the expert sample and half of participants (52%) in the APA sample considered that the new HD diagnosis should be included in the DSM-5 Main Manual, with the remaining participants being of the opinion that it should be included in an Appendix of Criteria Sets Provided for Further Study.
Discussion
We set out to empirically test the sensitivity, specificity and acceptability of the proposed DSM-5 diagnostic criteria for HD in two distinct samples of professionals using a clinical vignette methodology. While this approach cannot substitute a field trial with real patients, it provides a first approximation to this important question.
Overall, the sensitivity (or percentage of cases with HD identified correctly as such) of the diagnosis was high (approximately 80%) with even higher (approximately 90%) specificity (or percentage of cases with no HD identified correctly as such). This indicates that professionals find it particularly easy to rule out the new diagnosis when hoarding behaviour is better explained by other mental health problems. The sensitivity and specificity of the criteria were similar in the expert and non-expert (APA) samples, although the non-experts had somewhat lower sensitivity to detect the new diagnosis. This may be explained by the fact that most (>90%) declared not having expertise in OCD/hoarding. Increasing awareness and training clinicians to use the criteria will be important to further improve both the sensitivity and specificity of the new criteria. This will not only improve communication between professionals and between professionals and sufferers but also lead to more valid and reliable diagnoses for research.
A breakdown of the expert sample according to the participants' professional background, type of professional activity, level of seniority or whether they had substantial experience with hoarding cases revealed that this made no impact on the sensitivity/specificity of the criteria. This indicates that the criteria are generally well written and are easily understandable and applicable by a wide range of professionals.
As expected, the co-morbid OCD+HD and the paediatric HD cases (nos. 7 and 8 respectively) showed the lowest sensitivity and specificity. It appears that participants were slightly more reluctant to diagnose HD when OCD was present, even if the hoarding behaviour was clearly independent from the main OCD presentation. This is a critical issue as research has shown that, in some cases with OCD, hoarding is clearly secondary to the main OCD presentation (Mataix-Cols et al. Reference Mataix-Cols, Frost, Pertusa, Clark, Saxena, Leckman, Stein, Matsunaga and Wilhelm2010; Pertusa et al. Reference Pertusa, Frost, Fullana, Samuels, Steketee, Tolin, Saxena, Leckman and Mataix-Cols2010a, Reference Pertusa, Frost and Mataix-Colsb) (for example, the accumulation of trash due to contamination fears or fears of harming others). However, it is also clear that in other cases HD and OCD can co-occur as truly independent, i.e. co-morbid disorders. In these cases, the hoarding behaviour has all the features of HD and is not secondary to the OCD symptoms (Mataix-Cols et al. Reference Mataix-Cols, Frost, Pertusa, Clark, Saxena, Leckman, Stein, Matsunaga and Wilhelm2010; Pertusa et al. Reference Pertusa, Frost, Fullana, Samuels, Steketee, Tolin, Saxena, Leckman and Mataix-Cols2010a, Reference Pertusa, Frost and Mataix-Colsb). Our recommendation is that the text accompanying the HD criteria should be very explicit about the distinction between hoarding as a symptom of OCD and hoarding as a separate diagnosis. Some useful guidelines have already been put forward to facilitate the differential diagnosis between HD and OCD-related hoarding symptoms (see Mataix-Cols et al. Reference Mataix-Cols, Frost, Pertusa, Clark, Saxena, Leckman, Stein, Matsunaga and Wilhelm2010; Pertusa et al. Reference Pertusa, Frost and Mataix-Cols2010b). Again, training professionals to make the differential diagnosis will be essential. The development of relevant materials such as case vignettes, videos and semi-structured interviews such as the recently developed Structured Interview for Hoarding Disorder (Pertusa & Mataix-Cols, unpublished observations) should facilitate this endeavour.
The paediatric hoarding case (no. 8) was diagnosed correctly by 76% and 65% of the participants in the expert and APA samples, respectively. This could indicate that either the criteria are less suitable for younger sufferers or simply that most participants in this survey have not encountered paediatric hoarding cases in their clinical practice. Indeed, although based on retrospective report studies, HD is thought to originate in childhood/adolescence, the vast majority of cases seen for research and treatment are in their 40s–50s (Mataix-Cols et al. Reference Mataix-Cols, Frost, Pertusa, Clark, Saxena, Leckman, Stein, Matsunaga and Wilhelm2010). The literature on paediatric hoarding is extremely scant, with only a few cases having been described so far (Plimpton et al. Reference Plimpton, Frost, Abbey and Dorer2009). We conclude that further research into hoarding in childhood/adolescence is urgently needed. We also recommend that the DSM-5 field trials be extended to children/adolescents in order to test adequacy of the proposed HD criteria in this group. This work may lead to further improvements in the wording of the criteria.
The individual HD criteria showed acceptable sensitivity and specificity. The exclusion criteria F (‘Hoarding symptoms not restricted to the symptoms of another mental disorder’) and E (‘Hoarding symptoms not due to a general medical condition’) showed the lowest sensitivity and specificity. As suggested earlier, the text accompanying the criteria should be explicit regarding differential diagnoses and list all known possible mental disorders and medical conditions that can present with hoarding symptoms, with special emphasis on OCD.
The new diagnosis was deemed highly acceptable for both professionals and sufferers alike, with over 94% and 89% of participants in the expert and APA samples indicating that they thought the criteria would be very or at least somewhat acceptable for professionals and sufferers. This indicates the broad acceptability of the proposed disorder and is consistent with previous feedback received from both professionals and members of the public on the DSM-5 website.
Participants were more divided regarding the most appropriate placement of the new disorder in the forthcoming DSM-5. Whereas most (71%) participants in the expert sample indicated that it would be better placed in the main manual, only 52% in the APA sample supported such view. It is unclear why professionals from the APA sample were less supportive of the inclusion of the new diagnosis in the main manual. Our impression, based on the open responses to this survey and discussions with other professionals, is that those who have encountered large numbers of such cases in clinical practice tend to be highly supportive of the inclusion of HD in the main manual.
As discussed in detail elsewhere (Mataix-Cols et al. Reference Mataix-Cols, Frost, Pertusa, Clark, Saxena, Leckman, Stein, Matsunaga and Wilhelm2010), hoarding is inadequately covered in the current DSM-IV-TR as it is ambiguously listed as a criterion of obsessive-compulsive personality disorder (OCPD) and is not mentioned in the OCD section. Furthermore, it is now clear that most hoarding cases do not meet diagnostic criteria for either OCD or OCPD (Pertusa et al. Reference Pertusa, Fullana, Singh, Alonso, Menchon and Mataix-Cols2008, Reference Pertusa, Frost, Fullana, Samuels, Steketee, Tolin, Saxena, Leckman and Mataix-Cols2010a). We asked the study participants to diagnose one of the cases listed in the DSM-IV Casebook (pp. 15–17) entitled ‘Something of Value’ (Spitzer et al. Reference Spitzer, Gibbon, Skodol, Williams and First2002), strictly adhering to DSM-IV-TR criteria. According to the DSM-IV Casebook, the most appropriate diagnosis would be the residual category ‘Anxiety Disorder Not Otherwise Specified’ or, alternatively, ‘Personality Disorder Not Otherwise Specified’. A diagnosis of OCD cannot be made because the individual did not endorse any strictly defined obsessions or compulsions. Similarly, a diagnosis of OCPD cannot be made because this case did not endorse other features of OCPD other than hoarding. We found that 40–60% of participants diagnosed OCD, whereas approximately 30% indicated that the behaviour was pathological but that it did not fit into any of the current DSM-IV-TR categories. These results confirm that the existing DSM-IV-TR criteria are largely inadequate as they lead to an over-diagnosis of OCD and have low reliability. The addition of the new diagnosis in DSM-5 would resolve these issues.
The main limitation of the current study is the relatively low response rate. Only 30% of the invited experts and 10% of the APA sample completed their surveys. This raises the question of the representativeness of these samples. The response rate of the expert sample was similar, or even higher, to that obtained in other studies using similar methodology (Samuel & Widiger, Reference Samuel and Widiger2006; Lowe & Widiger, Reference Lowe and Widiger2009; Rottman et al. Reference Rottman, Ahn, Sanislow and Kim2009). On the other hand, the response rate of the APA sample was marginally lower than expected. The survey was relatively long and required some time and effort to complete. Furthermore, the APA sample was randomly selected from the member directory and presumably were less motivated to participate as OCD/hoarding was not their main area of interest. Finally, it is also likely that many of the email addresses were no longer in use or regularly accessed by their owners, so that the response rate in both samples may appear to be lower than it actually was.
To conclude, the proposed criteria for HD have high sensitivity and specificity. The criteria are also deemed acceptable for professionals and sufferers alike. Training of professionals and the development and validation of semi-structured diagnostic instruments should improve diagnostic accuracy even further. A field trial is now needed to confirm these encouraging findings with real patients in real clinical settings.
Note
Supplementary information accompanies this paper on the Journal's website (http://journals.cambridge.org/psm).
Acknowledgements
We thank our colleagues for their enthusiastic response to this survey. We also thank Professor Katharine Phillips, M.D., for her guidance during the preparation of this manuscript.
Declaration of Interest
David Mataix-Cols, Ph.D., is official advisor to the DSM-5 Obsessive-Compulsive Spectrum Sub-Work Group of the Anxiety, Obsessive-Compulsive Spectrum, Post-Traumatic, and Dissociative Disorders Work Group.
