Introduction
Major depressive disorder (MDD) is a common and important public health concern, with lifetime prevalences of approximately 13% in men and 24% in women (de Graaf et al. Reference de Graaf, tenHave, vanGool and van Dorsselaer2012). MDD is a genetically complex trait with a heritability of 31–42%, and it has a complex multifaceted etiology and pathophysiology (Major Depressive Disorder Working Group of the Psychiatric GWAS Consortium et al. Reference Ripke, Wray, Lewis, Hamilton, Weissman, Breen, Byrne, Blackwood, Boomsma, Cichon, Heath, Holsboer, Lucae, Madden, Martin, McGuffin, Muglia, Noethen, Penninx, Pergadia, Potash, Rietschel, Lin, Müller-Myhsok, Shi, Steinberg, Grabe, Lichtenstein, Magnusson, Perlis, Preisig, Smoller, Stefansson, Uher, Kutalik, Tansey, Teumer, Viktorin, Barnes, Bettecken, Binder, Breuer, Castro, Churchill, Coryell, Craddock, Craig, Czamara, De Geus, Degenhardt, Farmer, Fava, Frank, Gainer, Gallagher, Gordon, Goryachev, Gross, Guipponi, Henders, Herms, Hickie, Hoefels, Hoogendijk, Hottenga, Iosifescu, Ising, Jones, Jones, Jung-Ying, Knowles, Kohane, Kohli, Korszun, Landen, Lawson, Lewis, MacIntyre, Maier, Mattheisen, McGrath, McIntosh, McLean, Middeldorp, Middleton, Montgomery, Murphy, Nauck, Nolen, Nyholt, O'Donovan, Oskarsson, Pedersen, Scheftner, Schulz, Schulze, Shyn, Sigurdsson, Slager, Smit, Stefansson, Steffens, Thorgeirsson, Tozzi, Treutlein, Uhr, van den Oord, Van Grootheest, Völzke, Weilburg, Willemsen, Zitman, Neale, Daly, Levinson and Sullivan2013). It is the second leading cause of years lived with disability (Murray et al. Reference Murray, Vos, Lozano, AlMazroa and Memish2013), and has been related to increased onset of cardiometabolic diseases (Nicholson et al. Reference Nicholson, Kuper and Hemingway2006; Mezuk et al. Reference Mezuk, Eaton, Albrecht and Golden2008) and mortality (Cuijpers et al. Reference Cuijpers, Vogelzangs, Twisk, Kleiboer, Li and Penninx2014). Within research settings, the presence of MDD can be validly evaluated by (semi-) structured diagnostic interviews carried out by trained interviewers (Wittchen, Reference Wittchen1994; Sheehan et al. Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller, Hergueta, Baker and Dunbar1998). Because these are not always feasible in large samples and large international collaborations, briefer self-report depression instruments are often preferred. The large majority of depression instruments measure current depressive symptomatology instead of its longer-term burden. Measures of lifetime history of depression, however, can be of particular relevance in various fields of research, including psychosomatic studies examining the association of depression with (the onset of) somatic diseases, and biomarker studies. Strong calls for valid brief instruments for lifetime MDD status come from imaging studies, studies into co-morbidity of somatic and mental disorders and from genome-wide association studies (GWAS) for depression. GWAS with over 9000 MDD cases have not yet resulted in robustly replicated loci for MDD (Major Depressive Disorder Working Group of the Psychiatric GWAS Consortium et al. Reference Ripke, Wray, Lewis, Hamilton, Weissman, Breen, Byrne, Blackwood, Boomsma, Cichon, Heath, Holsboer, Lucae, Madden, Martin, McGuffin, Muglia, Noethen, Penninx, Pergadia, Potash, Rietschel, Lin, Müller-Myhsok, Shi, Steinberg, Grabe, Lichtenstein, Magnusson, Perlis, Preisig, Smoller, Stefansson, Uher, Kutalik, Tansey, Teumer, Viktorin, Barnes, Bettecken, Binder, Breuer, Castro, Churchill, Coryell, Craddock, Craig, Czamara, De Geus, Degenhardt, Farmer, Fava, Frank, Gainer, Gallagher, Gordon, Goryachev, Gross, Guipponi, Henders, Herms, Hickie, Hoefels, Hoogendijk, Hottenga, Iosifescu, Ising, Jones, Jones, Jung-Ying, Knowles, Kohane, Kohli, Korszun, Landen, Lawson, Lewis, MacIntyre, Maier, Mattheisen, McGrath, McIntosh, McLean, Middeldorp, Middleton, Montgomery, Murphy, Nauck, Nolen, Nyholt, O'Donovan, Oskarsson, Pedersen, Scheftner, Schulz, Schulze, Shyn, Sigurdsson, Slager, Smit, Stefansson, Steffens, Thorgeirsson, Tozzi, Treutlein, Uhr, van den Oord, Van Grootheest, Völzke, Weilburg, Willemsen, Zitman, Neale, Daly, Levinson and Sullivan2013), even though it has been estimated that common single nucleotide polymorphisms as studied in GWAS explain a substantial part (21%) of the liability to MDD (Lee et al. Reference Lee, Ripke, Neale, Faraone, Purcell, Perlis, Mowry, Thapar, Goddard, Witte, Absher, Agartz, Akil, Amin, Andreassen, Anjorin, Anney, Anttila, Arking, Asherson, Azevedo, Backlund, Badner, Bailey, Banaschewski, Barchas, Barnes, Barrett, Bass, Battaglia, Bauer, Bayes, Bellivier, Bergen, Berrettini, Betancur, Bettecken, Biederman, Binder, Black, Blackwood, Bloss, Boehnke, Boomsma, Breen, Breuer, Bruggeman, Cormican, Buccola, Buitelaar, Bunney, Buxbaum, Byerley, Byrne, Caesar, Cahn, Cantor, Casas, Chakravarti, Chambert, Choudhury, Cichon, Cloninger, Collier, Cook, Coon, Cormand, Corvin, Coryell, Craig, Craig, Crosbie, Cuccaro, Curtis, Czamara, Datta, Dawson, Day, de Geus, Degenhardt, Djurovic, Donohoe, Doyle, Duan, Dudbridge, Duketis, Ebstein, Edenberg, Elia, Ennis, Etain, Fanous, Farmer, Ferrier, Flickinger, Fombonne, Foroud, Frank, Franke, Fraser, Freedman, Freimer, Freitag, Friedl, Frisen, Gallagher, Gejman, Georgieva, Gershon, Geschwind, Giegling, Gill, Gordon, Gordon-Smith, Green, Greenwood, Grice, Gross, Grozeva, Guan, Gurling, de, Haines, Hakonarson, Hallmayer, Hamilton, Hamshere, Hansen, Hartmann, Hautzinger, Heath, Henders, Herms, Hickie, Hipolito, Hoefels, Holmans, Holsboer, Hoogendijk, Hottenga, Hultman, Hus, Ingason, Ising, Jamain, Jones, Jones, Jones, Tzeng, Kahler, Kahn, Kandaswamy, Keller, Kennedy, Kenny, Kent, Kim, Kirov, Klauck, Klei, Knowles, Kohli, Koller, Konte, Korszun, Krabbendam, Krasucki, Kuntsi, Kwan, Landen, Langstrom, Lathrop, Lawrence, Lawson, Leboyer, Ledbetter, Lee, Lencz, Lesch, Levinson, Lewis, Li, Lichtenstein, Lieberman, Lin, Linszen, Liu, Lohoff, Loo, Lord, Lowe, Lucae, MacIntyre, Madden, Maestrini, Magnusson, Mahon, Maier, Malhotra, Mane, Martin, Martin, Mattheisen, Matthews, Mattingsdal, McCarroll, McGhee, McGough, McGrath, McGuffin, McInnis, McIntosh, McKinney, McLean, McMahon, McMahon, McQuillin, Medeiros, Medland, Meier, Melle, Meng, Meyer, Middeldorp, Middleton, Milanova, Miranda, Monaco, Montgomery, Moran, Moreno-De-Luca, Morken, Morris, Morrow, Moskvina, Muglia, Muhleisen, Muir, Muller-Myhsok, Murtha, Myers, Myin-Germeys, Neale, Nelson, Nievergelt, Nikolov, Nimgaonkar, Nolen, Nothen, Nurnberger, Nwulia, Nyholt, O'Dushlaine, Oades and Olincy2013). One important reason for the lack of identified loci is that much larger samples are needed.
Within the Dutch arm of the Biobanking and Biomolecular Resources Research Infrastructure (BBMRI-NL), we started a project that aimed to assess the presence of lifetime MDD in a standardized way in tens of thousands persons who have already participated in GWAS, transcriptomics and metabolomics cohort studies (Boomsma et al. Reference Boomsma, Wijmenga, Slagboom, Swertz, Karssen, Abdellaoui, Ye, Guryev, Vermaat, van, Francioli, Hottenga, Laros, Li, Li, Cao, Chen, Du, Li, Cao, van, Menelaou, Pulit, Hehir-Kwa, Beekman, Elbers, Byelas, de Craen, Deelen, Dijkstra, den Dunnen, de, Houwing-Duistermaat, Koval, Estrada, Hofman, Kanterakis, Enckevort, Mai, Kattenberg, van Leeuwen, Neerincx, Oostra, Rivadeneira, Suchiman, Uitterlinden, Willemsen, Wolffenbuttel, Wang, de Bakker, van Ommen and van Duijn2014; Zhernakova et al. Reference Zhernakova, Deelen, Vermaat, van Iterson, van Galen, Arindrato, van t Hof, Mei, van Dijk, Westra, Bonder, van Rooij, Verkerk, Jhamai, Moed, Kielbasa, Bot, Nooren, Pool, van Dongen, Hottenga, Stehouwer, van der Kallen, Schalkwijk, Zhernakova, Li, Tigchelaar, Beekman, Deelen, van Heemst, van den Berg, Hofman, Uitterlinden, van Greevenbroek, Veldink, Boomsma, van Duijn, Slagboom, Swertz, Isaacs, van Meurs, Jansen, Heijmans, Hoen and Franke2015). For this purpose, an online self-report depression questionnaire was developed (LIfetime Depression Assessment Self-report; LIDAS), which was largely based on the Composite International Diagnostic Interview (CIDI) short-form for lifetime depression (CIDI-SF; Kessler et al. Reference Kessler, Andrews, Mroczek, Ustun and Ittchen1998; Hamilton et al. Reference Hamilton, Strader, Pratt, Maiese, Hendershot, Kwok, Hammond, Huggins, Jackman, Pan, Nettles, Beaty, Farrer, Kraft, Marazita, Ordovas, Pato, Spitz, Wagener, Williams, Junkins, Harlan, Ramos and Haines2011). The CIDI-SF instrument is the instrument of choice for depression in the Phenx toolbox, a catalog of recommended, standard measures of phenotypes and exposures for use in biomedical research, which facilitates cross-study analysis (Hamilton et al. Reference Hamilton, Strader, Pratt, Maiese, Hendershot, Kwok, Hammond, Huggins, Jackman, Pan, Nettles, Beaty, Farrer, Kraft, Marazita, Ordovas, Pato, Spitz, Wagener, Williams, Junkins, Harlan, Ramos and Haines2011). Furthermore, the online instrument has additional items on MDD symptoms to cover all nine Diagnostic and Statistical Manual of Mental Disorders (DSM) criteria for MDD, and items on psychiatric diagnosis, treatment history, sociodemography and lifestyle.
Establishing the reliability and validity of new instruments and assessment modes is important. In validation studies of online current depressive symptomatology instruments, the estimated sensitivity and specificity ranged from 25 to 86 and from 59 to 89%, respectively (Carlbring et al. Reference Carlbring, Forslin, Ljungstrand, Willebrand, Strandlund, Ekselius and Andersson2002; Farvolden et al. Reference Farvolden, McBride, Bagby and Ravitz2003; Lin et al. Reference Lin, Bai, Liu, Hsiao, Chen, Tsai, Ouyang, Wu and Li2007; Donker et al. Reference Donker, van, Marks and Cuijpers2009; Nguyen et al. Reference Nguyen, Klein, Meyer, Austin and Abbott2015). Few studies assessed the feasibility and psychometric properties of online lifetime MDD measures (Kendler et al. Reference Kendler, Myers, Potter and Opalesky2009; Sanders et al. Reference Sanders, Levinson, Duan, Dennis, Li, Kendler, Rice, Shi, Mowry, Amin, Silverman, Buccola, Byerley, Black, Freedman, Cloninger and Gejman2010; Shimoda et al. Reference Shimoda, Inoue, Tsuno and Kawakami2015). Shimoda et al. (Reference Shimoda, Inoue, Tsuno and Kawakami2015) tested the 1-year test–retest reliability for a modified online version of the World Health Organization (WHO) CIDI for MDD in a working population with a low prevalence of depression. Although in the total sample the concordance rates were high, concordance rates were low (κ 0.15) for persons diagnosed with depression at one or two of the time points (Shimoda et al. Reference Shimoda, Inoue, Tsuno and Kawakami2015). Two studies with CIDI-SF- or DSM-IV-based online instruments yielded higher MDD prevalence rates than reported in population-based studies using structured interviews (Kendler et al. Reference Kendler, Myers, Potter and Opalesky2009; Sanders et al. Reference Sanders, Levinson, Duan, Dennis, Li, Kendler, Rice, Shi, Mowry, Amin, Silverman, Buccola, Byerley, Black, Freedman, Cloninger and Gejman2010).
The present study aims to evaluate the (1) feasibility, (2) estimated population MDD prevalence, and (3) diagnostic accuracy (sensitivity and specificity) of the online LIDAS in subsamples of two Dutch cohorts: the Netherlands Study of Depression and Anxiety (NESDA) (Penninx et al. Reference Penninx, Beekman, Smit, Zitman, Nolen, Spinhoven, Cuijpers, De Jong, Van Marwijk, Assendelft, Van der Meer, Verhaak, Wensing, De Graaf, Hoogendijk, Ormel and Van Dyck2008) and the Netherlands Twin Register (NTR) (Boomsma et al. Reference Boomsma, Willemsen, Sullivan, Heutink, Meijer, Sondervan, Kluft, Smit, Nolen, Zitman, Smit, Hoogendijk, van, de Geus and Penninx2008).
Method
Participants were recruited from two ongoing studies (NESDA and NTR). Participants of NESDA and a subgroup of NTR completed the index test (verbal CIDI) as part of earlier data collections in these studies.
NESDA
NESDA consists of 2981 participants aged 18–65 years, comprising persons with and without depressive and/or anxiety disorders. Baseline interviews were conducted from 2004 to 2007, and follow-up interviews took place after 2, 4, 6 and 9 years (n = 2596, n = 2402, n = 2256, and still ongoing, respectively). During all these assessments, the presence of depressive disorders (MDD and dysthymia) and anxiety disorders was ascertained with the DSM-IV-based CIDI (version 2.1) by specially trained research staff. A detailed description of the NESDA design can be found elsewhere (Penninx et al. Reference Penninx, Beekman, Smit, Zitman, Nolen, Spinhoven, Cuijpers, De Jong, Van Marwijk, Assendelft, Van der Meer, Verhaak, Wensing, De Graaf, Hoogendijk, Ormel and Van Dyck2008). The NESDA research protocol was approved by the ethical committee of the participating centers. All participants provided written informed consent.
The 9-year follow-up was scheduled for 2014–2016. For the validation study, we consecutively selected all active NESDA participants who completed the 9-year follow-up assessment between 1 February 2014 and 31 October 2014 (n = 355). We classified participants of this subsample as either controls (no lifetime depressive disorder at the 9 year follow-up, n = 133) or lifetime MDD case (n = 222) according to face-to-face DSM-IV-based CIDI interviews.
NTR
NTR comprises ongoing twin-family studies on health-related behavior and assesses families with adolescent and (young) adult twins since 1991 (Boomsma et al. Reference Boomsma, Vink, van Beijsterveldt, de Geus, Beem, Mulder, Derks, Riese, Willemsen, Bartels, van den Berg, Kupper, Polderman, Posthuma, Rietveld, Stubbe, Knol, Stroet and van Baal2002). Participants are invited every 2 to 3 years to complete a survey that contains questions about health, lifestyle, personality and psychopathology. The study was approved by the medical ethics committee, and all participants signed informed consent forms. For NTR, we recruited 398 participants in total for the validation study from two groups.
The first group (lifetime MDD cases) consisted of a random sample of 100 adults with a lifetime MDD diagnosis according to DSM-IV criteria based on the CIDI. This CIDI took place by telephone in 1997 and in 2007 in an NTR subsample (Middeldorp et al. Reference Middeldorp, Birley, Cath, Gillespie, Willemsen, Statham, de Geus, Andrews, van, Beem, Sullivan, Martin and Boomsma2005a , Reference Middeldorp, Cath, van den Berg, Beem, Van Dyck, Boomsma and Canli b ; Boomsma et al. Reference Boomsma, Willemsen, Sullivan, Heutink, Meijer, Sondervan, Kluft, Smit, Nolen, Zitman, Smit, Hoogendijk, van, de Geus and Penninx2008).
The second group (population-based sample) was a sample of 298 NTR participants randomly selected from all adult NTR participants irrespective of their depression scores. All selected individuals participated in NTR research during the last 4 years. To enable analyses of differences in participation between men and women and between younger and older adults, the selection was balanced in such a way that sex and age (below and above 30 years) were evenly distributed.
Reference index MDD
The CIDI interviews, conducted by trained interviewers in NESDA and NTR, provided the index lifetime MDD diagnoses and served as reference index for the online LIDAS instrument. The CIDI instrument has a test–retest agreement of 92% (test–retest κ 0.71) and an inter-rater agreement of 98% (inter-rater κ 0.95) for any DSM-III depressive disorder (Wittchen, Reference Wittchen1994). Lifetime MDD cases were defined as having a DSM-based lifetime MDD according to the CIDI. Both NESDA and NTR provided lifetime MDD cases. Lifetime controls were derived from the NESDA subsample only, and were the participants without a CIDI lifetime depressive disorder at the most recent NESDA wave (9-year follow-up). Participants approached to complete the online LIDAS instrument were not told whether they fulfilled the criteria of MDD according to the index CIDI.
Procedure LIDAS assessment
In both NESDA and NTR, an invitation letter was mailed by post to complete the online LIDAS. A postal reminder was sent 2 weeks later. Furthermore, for those who had previously provided their email addresses, an invitation letter and reminder by email were sent 4 days after the postal invitation and the postal reminder. All letters contained personal URLs (Uniform Resource Locators) and four-digit passwords to access the online LIDAS.
To maximize the sample for the validation study, NESDA participants who did not want to participate online could call the research office to complete the LIDAS by means of a telephone interview conducted by a trained research assistant. For NTR, persons who did not want to participate online could complete the LIDAS by means of a paper version of the questionnaire, which was sent along with the reminder. For NESDA, the online data were collected between 6 February 2015 and 19 March 2015, and for NTR between 18 February 2015 and 30 June 2015.
MDD measurement with LIDAS
The online LIDAS was based on the CIDI-SF (Kessler et al. Reference Kessler, Andrews, Mroczek, Ustun and Ittchen1998), but was extended to cover all nine DSM criteria of MDD (Wittchen, Reference Wittchen1994). Furthermore, questions on sociodemographics, lifestyle and other psychiatric disorders were asked. From a list of 13 psychiatric disorders and addictions (e.g. depression, bipolar disorder, anxiety disorder, alcohol addiction), participants could select the ones that they had ever been diagnosed with, and subsequently the ones they ever received treatment for. Online Supplementary Table S1 shows the questions concerning MDD criteria and former psychiatric diagnoses and treatment. The LIDAS was programmed for online use and consisted of (a maximum of) 35 questions. Questions were presented successively, with preprogrammed skip patterns for items that were not applicable. For example, if a participant did not fulfill the two main core symptoms of MDD (depressed mood and anhedonia), the questions related to the other MDD symptoms were not applicable anymore and therefore not asked. Items had forced choice response formats to reduce the number of missing values. Participants were offered the possibility of completing the survey in multiple stages, meaning that they could temporarily stop and continue later at that point.
We used three algorithms to determine MDD status from the LIDAS and compared test performance of these three algorithms. In line with the original CIDI-SF scoring algorithm, we only used the symptom criteria (the A criteria as listed in the DSM-IV). First, based on the presence of the MDD symptoms measured in the LIDAS, we classified participants as either having a lifetime MDD (at least five out of nine MDD symptoms, including at least one core symptoms of MDD, i.e. depressed mood or anhedonia) or no lifetime MDD (no core symptom of MDD or less than five MDD symptoms). Second, we expanded the definition of lifetime MDD status with LIDAS data on self-reported diagnosis or treatment of depression, and antidepressant medication use. Cases were defined as having either (1) at least five out of nine MDD symptoms, including at least one core symptoms of MDD, or (2) self-reported diagnosis or treatment of depression in lifetime, or antidepressant medication use in lifetime. All other participants were classified as controls. Third, since it may be important to have controls free of any psychiatric disorders, we additionally excluded controls with a diagnosis or treatment of psychiatric disorders other than MDD to define a ‘pure control’ group. These algorithms are shown in online Supplementary Table S2. The CIDI-SF only asks about eight MDD symptom criteria, whereas we chose to use all nine MDD criteria in the scoring algorithms. This better reflects the concept of MDD, and provides opportunities for future analyses with, for example, symptom subclasses and number of DSM-symptoms, which were shown to be relevant for picking up larger or more specific genetic vulnerability (Milaneschi et al. Reference Milaneschi, Lamers, Peyrot, Abdellaoui, Willemsen, Hottenga, Jansen, Mbarek, Dehghan, Lu, Boomsma and Penninx2016; Verduijn et al. Reference Verduijn, Milaneschi, Peyrot, Hottenga, Abdellaoui, de Geus, Smit, Breen, Lewis, Boomsma, Beekman and Penninx2016).
After completing the online questionnaire, persons were asked to complete evaluation questions on the clarity of the instructions, questions, answers (clear v. unclear), and user-friendliness and lay-out (rating between 1 and 10). In addition, the time needed to complete the online questionnaire was automatically recorded.
Statistical analyses
Statistical analyses were conducted in SPSS for Windows version 20.0 (IBM Corp., USA). First, characteristics, participation rates, and evaluation scores from the sample were described. We compared age, sex and education level between (1) participants and non-participants, and (2) completers and non-completers, using independent t tests or χ2 tests where appropriate.
Second, for the population sample without index CIDI data, the prevalence of MDD by the LIDAS was calculated. Furthermore, we studied whether the presence of MDD in this sample was related to sex and age using χ2 tests and independent-samples t tests, respectively.
Third, we pooled data of the NESDA sample (index MDD cases and controls) and NTR index cases and calculated the sensitivity and specificity of the LIDAS MDD instrument, using the index CIDI as reference. The 95% confidence intervals (CIs) for the sensitivity and specificity were calculated using bootstrapping procedures based on 10 000 resamples. We also calculated Youden's J. Youden's J provides a summary measure of the performance of a diagnostic test giving equal weight to false positives and false negatives, and was calculated as: sensitivity + specificity – 1 (Youden, Reference Youden1950). A Youden's J of 1 would indicate a perfect test (no false positives or false negatives). Accuracy was calculated as (sensitivity + specificity)/2. Finally, we tested the agreement on the LIDAS age-of-onset question and number of episodes question (single v. multiple) with the corresponding data of the index CIDI by calculating the intra-class correlation coefficient and Cohen's κ, respectively.
Results
Feasibility
Fig. 1 shows the flowchart of the study. In total, 753 persons were invited to complete the LIDAS (322 index MDD cases, 133 index controls and 298 with no index CIDI). Of these, 541 persons participated (72%), and 509 persons (68%) completed sufficient items to determine MDD status with the LIDAS (177 index MDD cases, 87 index controls and 245 persons with no index CIDI). The majority (n = 419, 82%) completed the LIDAS online. Persons who participated did not differ from non-participants in terms of age (p = 0.21), sex (p = 0.53) and education level (p = 0.19). Similarly, persons who completed sufficient items to determine MDD status (i.e. completers) did not differ from non-completers in age (p = 0.31), sex (p = 0.16) and education level (p = 0.21).
Flowchart of the participants. NESDA, Netherlands Study of Depression and Anxiety; NTR, Netherlands Twin Register; CIDI, Composite International Diagnostic Interview; MDD, major depressive disorder; LIDAS, LIfetime Depression Assessment Self-report.
Table 1 shows the baseline characteristics of the sample who completed the LIDAS (n = 509) stratified by group. Overall, the mean age was 49.2 (s.d. = 15.9) years, and 42% were male. The median time to complete the online questionnaire (including the evaluation questions) was 6.19 min (interquartile range 3.59–9.37 min).The large majority found the questions and answer possibilities clear (98% and 96%, respectively). Participants gave, on a scale from 0 to 10, a median score of 8 (interquartile range 7–9) for the lay-out, and 8 (interquartile range 8–9) for user-friendliness.
Descriptive characteristics of the sample who completed the online LIDAS (n = 509)
LIDAS, LIfetime Depression Assessment Self-report; MDD, major depressive disorder; s.d., standard deviation; IQR, interquartile range; NESDA, Netherlands Study of Depression and Anxiety; NTR, Netherlands Twin Register.
a NESDA participants were offered to complete the LIDAS by a telephone interview, whereas NTR participants were offered to complete the LIDAS on paper.
Population-based prevalence
In the population-based sample (n = 245), the prevalence of lifetime MDD according to the LIDAS definitions ranged from 19.6% (algorithm 1) to 20.8% (algorithms 2 and 3). In the population-based sample, the prevalence of lifetime MDD was slightly lower in males (17.6 to 18.4% across the three algorithms) than in females (22.0 and 23.7% across the three algorithms) but this difference was not statistically significant (all p’s > 0.05, data not shown). In addition, lifetime MDD status was not associated with age (all p’s > 0.05; data not shown).
Sensitivity and specificity
A total of 177 index MDD cases, and 87 index controls completed both the index instrument and the LIDAS. Table 2 shows the results of the sensitivity and specificity analysis, using the index MDD status as reference. When measuring lifetime MDD based on the presence of LIDAS MDD symptoms only, good specificity (0.86, 95% CI 0.78–0.93) but rather low sensitivity (i.e. high number of false negatives; 0.66, 95% CI 0.59–0.73) compared with the index CIDI was found (accuracy: 0.73, Youden's J: 0.52). However, when the three questions about lifetime depression diagnosis and treatment were also taken into account (algorithm 2), adequate levels of both sensitivity and specificity (0.80, 95% CI 0.73–0.85; and 0.82, 95% CI 0.74–0.90, respectively) were found compared with the index CIDI for MDD. Overall, algorithm 3 (algorithm 2 with removal of controls with other psychiatric disorders) had the highest Youden's J (0.66) and accuracy (0.84), with sensitivity levels of 0.85 (95% CI 0.80–0.91) and specificity levels of 0.80 (95% CI 0.72–0.89) compared with the reference index.
Sensitivity and specificity of three different lifetime MDD classifications of the online LIDAS compared with index CIDI diagnosis of MDD
MDD, Major depressive disorder; LIDAS, LIfetime Depression Assessment Self-report; CIDI, Composite International Diagnostic Interview; L+, MDD according to the LIDAS; I+, MDD according to index CIDI; I−, no MDD according to index CIDI; L−, no MDD according to LIDAS; CI, confidence interval.
a Total n = 264. For algorithm 3, the n is smaller because persons with other psychiatric disorders (measured with LIDAS) were excluded from the controls.
Because persons with lifetime anxiety disorders were over-represented in the index controls due to the oversampling of depressive and anxiety disorders in the NESDA cohort, we conducted additional analyses in which we removed persons with an index lifetime anxiety disorder from the index controls (n = 23), leaving 64 index MDD controls. In this additional analysis, the specificity of algorithm 3 improved to 85% (95% CI 76–94%; data not shown). Furthermore, when we restricted our sensitivity/specificity analyses to the participants who completed the LIDAS online (n = 230), we found a sensitivity and specificity that were very similar to our main analysis (data not shown). The intra-class correlation coefficient for the age-of-onset question from the index CIDI and LIDAS was 0.64 (95% CI 0.52–0.73; n = 131). The agreement of number of episodes (single v. multiple) measured in the index CIDI and the LIDAS (n = 132) was as follows: 15 persons consistently reported a single episode, and 64 persons consistently reported multiple episodes, 43 persons had a single episode in the CIDI but multiple episodes in the LIDAS, and 10 persons had multiple episodes in the CIDI but a single episode in the LIDAS. This yielded a κ of 0.13 (95% CI −0.01 to 0.28).
Discussion
There is a paucity of valid, brief instruments for measuring lifetime MDD status that can be used in large cohort studies, such as needed for GWAS for depression. This study aimed to test the validity and feasibility of a brief online instrument assessing lifetime MDD using CIDI MDD interviews as reference, in participants of two Dutch cohort studies. Regarding validity, we found that a combination of MDD symptoms and self-reported depression diagnosis and treatment resulted in adequate levels of both sensitivity and specificity (⩾0.80) compared with the index CIDI for MDD. In the population-based sample, LIDAS qualified approximately a fifth as having a lifetime history of MDD. Although females had slightly higher lifetime MDD diagnosis than males, this difference was not significant. Regarding feasibility, more than two-thirds of those invited completed the self-reported LIDAS instrument, and over 80% did this online.
There are only a few studies on online instruments measuring lifetime MDD that assessed the psychometric properties (Kendler et al. Reference Kendler, Myers, Potter and Opalesky2009; Sanders et al. Reference Sanders, Levinson, Duan, Dennis, Li, Kendler, Rice, Shi, Mowry, Amin, Silverman, Buccola, Byerley, Black, Freedman, Cloninger and Gejman2010; Shimoda et al. Reference Shimoda, Inoue, Tsuno and Kawakami2015), but these studies did not include a reference diagnostic standard and therefore did not assess diagnostic accuracy. With respect to psychometric studies on paper versions of self-report lifetime MDD instruments, it has been reported that the lifetime version of the 22-item self-report Inventory to Diagnose Depression (IDD) had a sensitivity of 74% and a specificity of 93% for depression as compared with the Diagnostic Interview Schedule (DIS) (Zimmerman & Coryell, Reference Zimmerman and Coryell1987). Another study using a paper version of the IDD for lifetime MDD found even higher accuracy compared with the DIS (sensitivity: 83%, specificity: 100%). However, MDD cases and controls were not from the same source population, which may have biased these values (MDD cases were out-patients seeking treatment, whereas controls were selected from the working population) (Sakado et al. Reference Sakado, Sato, Uehara, Sato and Kameda1996).
Our sensitivity and specificity compare favorably with reports from validation studies of other online instruments for current depression symptomatology using a form of clinical diagnosis of depression as the reference index. Overall, in these studies the sensitivity levels ranged from 25 to 86%, and specificity levels were between 59 and 89% (Carlbring et al. Reference Carlbring, Forslin, Ljungstrand, Willebrand, Strandlund, Ekselius and Andersson2002; Farvolden et al. Reference Farvolden, McBride, Bagby and Ravitz2003; Lin et al. Reference Lin, Bai, Liu, Hsiao, Chen, Tsai, Ouyang, Wu and Li2007; Donker et al. Reference Donker, van, Marks and Cuijpers2009; Nguyen et al. Reference Nguyen, Klein, Meyer, Austin and Abbott2015). A small-scaled study with 53 participants found that the online CIDI-SF had a sensitivity of only 25%, and a specificity of 76% compared with current diagnosis of MDD according to the Structured Clinical Interview for DSM Disorders (SCID). These levels, in particular the sensitivity, are much lower than the values we found. This difference may be explained by the fact that we had a different reference instrument (CIDI), studied lifetime MDD status, and applied a more advanced algorithm for classifying our cases than the original CIDI-SF scoring method (Carlbring et al. Reference Carlbring, Forslin, Ljungstrand, Willebrand, Strandlund, Ekselius and Andersson2002). When we applied the original CIDI-SF scoring algorithm to the LIDAS and compared these with the CIDI, we found a lower sensitivity (0.69, 95% CI 0.62–0.76) and comparable specificity (0.84, 95% CI 0.76–0.91; data not shown) than the algorithm used in this paper. Our diagnostic accuracy levels for lifetime MDD are more comparable with that of the e-PASS instrument (sensitivity 0.86, specificity 0.79, reference Mini International Neuropsychiatric Interview; MINI-plus) (Nguyen et al. Reference Nguyen, Klein, Meyer, Austin and Abbott2015), and the Web-Based Depression and Anxiety Test (sensitivity 0.79, specificity 0.89, SCID) (Farvolden et al. Reference Farvolden, McBride, Bagby and Ravitz2003) for current MDD status.
For depression assessments in large cohort studies, both high sensitivity and specificity are of importance, because it is evenly important to validly identify MDD cases as it is to validly rule out MDD in controls. The top performance in terms of highest combined sensitivity and specificity was found for the algorithm in which lifetime MDD cases were defined as persons that fulfilled the MDD symptom criteria or had a diagnosis of – or treatment for – depression (including antidepressant medication use). Controls were defined as not fulfilling the above-mentioned criteria, and having no self-reported lifetime diagnosis or treatment for other psychiatric disorders. Although this means that some persons fall outside the classification of lifetime MDD cases and controls, this is probably a minority because other psychiatric disorders have either a low prevalence, or a high co-morbidity with MDD. In our study, 4% of the population-based sample fell outside the case–control classifications.
When measuring lifetime MDD based on the presence of LIDAS MDD symptoms only, good specificity but rather low sensitivity compared with the index CIDI was found, suggesting that a substantial number of persons with an index CIDI MDD will not be detected by measuring symptoms in lifetime only. This may be due to recall problems, as people may not correctly remember the past presence of each of these symptoms. However, when the three questions about diagnosis and treatment for depression – which may suffer less from recall issues – were additionally taken into account the sensitivity for depression became adequate.
The agreement of the questions on age of onset was quite good but that on the number of episode questions was low. The low agreement might be partly explained by the large time interval between the index CIDI and LIDAS assessment which could have caused increases in the number of episodes. Indeed, over 80% of the inconsistencies in reported episodes resulted from reporting single episodes in the CIDI and multiple episodes in the LIDAS. Our findings are in line with Bromet et al. (Reference Bromet, Dunn, Connell, Dew and Schulberg1986) where in women a semi-structured interview for MDD was used at baseline and 18 months later. Bromet et al. (Reference Bromet, Dunn, Connell, Dew and Schulberg1986) found correlations of 0.51 and 0.16 for age of onset and number of episodes measured 18 months apart, respectively. Moreover, unpublished data of the baseline and 2-year follow-up face-to-face CIDI from the NESDA study showed a correlation of r = 0.56 for age of onset (n = 763), and a Cohen's κ of 0.13 (95% CI 0.09–0.16) for single v. multiple episodes (M Bot et al., unpublished observations), illustrating the moderate agreement for age of onset and the poor agreement for multiple episodes even in regular diagnostic interviews.
These and other reports suggest that it is difficult to obtain reliable reports for number of episodes in particular (Bromet et al. Reference Bromet, Dunn, Connell, Dew and Schulberg1986; Simon et al. Reference Simon and Von Korff1995), and this MDD characteristic in the LIDAS should be interpreted with caution. Furthermore, age at onset might be a more important clinical characteristic than number of episodes in genetic studies: a recent study suggests that young age of onset is among the clinical characteristics that increases the genetic risk of depression (estimated by genome-wide genomic profile risk scores), whereas no consistent association was found for recurrent episodes (Verduijn et al. Reference Verduijn, Milaneschi, Peyrot, Hottenga, Abdellaoui, de Geus, Smit, Breen, Lewis, Boomsma, Beekman and Penninx2016).
The prevalence of lifetime MDD according to our three MDD algorithms was between 19 and 21%, which is similar to the prevalence of lifetime MDD in the Dutch population (19%) (de Graaf et al. Reference de Graaf, tenHave, vanGool and van Dorsselaer2012), and falls within the range of median prevalence levels found in countries participating in the WHO's World Mental Health Survey initiative (Kessler et al. Reference Kessler, Angermeyer, Anthony, de, Demyttenaere, Gasquet, de, Gluzman, Gureje, Haro, Kawakami, Karam, Levinson, Medina Mora, Oakley Browne, Posada-Villa, Stein, Adley Tsang, Aguilar-Gaxiola, Alonso, Lee, Heeringa, Pennell, Berglund, Gruber, Petukhova, Chatterji and Ustun2007b ). In contrast, two other studies with different online self-report lifetime MDD instruments (based on the CIDI-SF or DSM-IV) tended to result in higher prevalence rates of MDD than reported in population-based studies (Kendler et al. Reference Kendler, Myers, Potter and Opalesky2009; Sanders et al. Reference Sanders, Levinson, Duan, Dennis, Li, Kendler, Rice, Shi, Mowry, Amin, Silverman, Buccola, Byerley, Black, Freedman, Cloninger and Gejman2010). Whereas the overall population prevalence of MDD in our study was in line with other population-based studies, the prevalence did not differ significantly between females and males. This discrepancy with the known sex imbalance in MDD should be further studied. Age was not related to an increased risk of lifetime MDD. This might be due to the early onset of MDD in life (Kessler et al. Reference Kessler, Amminger, Aguilar-Gaxiola, Alonso, Lee and Ustun2007a ). In addition, studies paradoxically show that the prevalence of MDD appears to decline with higher age (Streiner et al. Reference Streiner, Patten, Anthony and Cairney2009).
Strengths of the study are the selection of a population that is representative for the prospective users of the LIDAS, as we will use this instrument in existing cohort studies with GWAS data already available. Although we oversampled MDD cases to enable diagnostic accuracy testing, we also included a population-based sample to estimate the population prevalences of MDD according to the online instrument. A limitation is that we used a verbal CIDI as the ‘gold standard’ reference. This does not equal diagnosis from clinicians. However, the CIDI has a high reliability and validity for the assessment of depressive disorders (Wittchen, Reference Wittchen1994). Second, we did not assess the test–retest reliability. Shimoda et al. (Reference Shimoda, Inoue, Tsuno and Kawakami2015) found that the 1-year test–retest reliability for the modified online version of the WHO CIDI for MDD was low in persons diagnosed with depression, although overall the reliability was high (Shimoda et al. Reference Shimoda, Inoue, Tsuno and Kawakami2015). Third, there was a time interval between the completion of the index CIDI and the online instrument. This should not affect the number of lifetime MDD cases (once a case, always a lifetime case), but controls could have developed a depressive episode in between. However, this number is likely to be small because this interval was at maximum 1 year and controls were free of MDD at each of the five measurement rounds over a period of 9 years. Moreover, if some persons would have developed an MDD episode between the index test and online testing, our estimate of the specificity (⩾0.80) is a conservative one. Finally, care should be taken to generalize the results to older adults as assessment of lifetime MDD might suffer from recall bias, because persons might forget about episodes or nowadays interpret prior symptoms more positively (Streiner et al. Reference Streiner, Patten, Anthony and Cairney2009). Lifetime assessment of MDD has its limitations due to such recall biases, but for genetic analysis it is an important construct, in particular as it also helps to rule out MDD diagnosis in controls.
In conclusion, we found adequate levels for the sensitivity and specificity of the online instrument to assess lifetime MDD status as compared with face-to-face interviews in adults. The legibility and presentation of the questions and answers were rated positive, and the completion time was short, yielding a good response rate of 68%. This shows that assessing lifetime MDD with this new online instrument is highly feasible. The LIDAS is therefore a promising avenue to quickly determine lifetime MDD status in large cohorts, such as needed for GWAS for depression.
Supplementary material
The supplementary material for this article can be found at http://dx.doi.org/10.1017/S0033291716002312
Acknowledgements
This research was financially supported by BBMRI-NL, a Research Infrastructure financed by the Dutch government (NWO 184.021.007). We would like to acknowledge all researchers involved in BBMRI-NL project 'Phenomics 2.0 - proof of principle for major depressive disorder'. The infrastructure for the NESDA study (www.nesda.nl) has been funded through the Geestkracht program of the Netherlands Organization for Health Research and Development (Zon-Mw, grant number 10-000-1002) and participating universities (VU University Medical Center, Leiden University Medical Center, University Medical Center Groningen).
Declaration of Interest
M.B., C.M.M., E.J.C.G., H.M.L., M.S., B.N., J.H.S. and D.I.B. report no conflicts of interest. B.W.J.H.P. has received research funding from Johnson & Johnson.
