Introduction
During the past decades, depression treatment has shifted from an approach based on clinical expertise towards an evidence-based approach using results from randomized clinical trials (RCTs) on antidepressants and/or psychotherapy (Fava et al. Reference Fava, Ruini and Rafanelli2005; IJff et al. Reference IJff, Huijbregts, van Marwijk, Beekman, Hakkaart-van Roijen, Rutten, Unutzer and van der Feltz-Cornelis2007). However, for methodological and ethical reasons, antidepressant efficacy trials (AETs) will always need strict, randomized and placebo-controlled conditions and use of stringent inclusion and exclusion criteria for patient selection. In this way, internal validity is optimized. However, by optimizing internal validity, external validity (generalizability) might be compromised. Hence, the generalizability of the results from AETs to clinical practice can be questioned (Stewart et al. Reference Stewart, McGrath and Quitkin1992; Licht et al. Reference Licht, Gouliaev, Vestergaard and Frydenberg1997; Wells, Reference Wells1999; Mulder et al. Reference Mulder, Frampton, Joyce and Porter2003; Tunis et al. Reference Tunis, Stryer and Clancy2003). Three studies in the United States examined the eligibility of depressive patients for inclusion in AETs (Zimmerman et al. Reference Zimmerman, Posternak and Chelminski2002b, Reference Zimmerman, Mattia and Posternak2004a; Zetin & Hoepner, Reference Zetin and Hoepner2007). These reported that only 12–34% of these patients were eligible for AETs. However, these investigations regarded only fee-for-service settings, which may not be generalizable to the European healthcare system. In a European study, the eligibility of volunteers for AETs was also found to be limited; 34% of the patients who volunteered for an AET finally entered the trial (Partonen et al. Reference Partonen, Sihvo and Lonnqvist1996). The majority of the volunteers were excluded because of co-morbid disorders. Investigators of the STAR*D trial (Rush et al. Reference Rush, Fava, Wisniewski, Lavori, Trivedi, Sackeim, Thase, Nierenberg, Quitkin, Kashner, Kupfer, Rosenbaum, Alpert, Stewart, McGrath, Biggs, Shores-Wilson, Lebowitz, Ritz and Niederehe2004) used less stringent inclusion criteria in order to obtain more generalizable results. In their study, 22% of the included patients would have been eligible for AETs and had a better treatment outcome than non-eligible patients (Wisniewski et al. Reference Wisniewski, Rush, Nierenberg, Gaynes, Warden, Luther, McGrath, Lavori, Thase, Fava and Trivedi2009). However, the generalizability of STAR*D to routine clinical practice may still be limited due to exclusion of prior non-responders to the study drugs and the use of a minimum baseline severity. In addition, the generalizability of STAR*D to non-US health settings is unclear (Hatcher, Reference Hatcher2008). In the present study, we investigated, in routine out-patient care, to what proportion of depressive patients the results of AETs would apply. We chose to limit the AETs to classical RCTs, since in national and international treatment guidelines (Karasu et al. Reference Karasu, Gelenberg, Merriam and Wang2000;National Taskforce Guideline, 2005;Anderson et al., Reference Anderson, Pilling, Barnes, Bayliss and Bird2009), classical RCTs are considered the most robust evidence for efficacy. We do, however, expect that, in the near future, the results from more pragmatic trials such as STAR*D and GENDEP (Rush et al. Reference Rush, Fava, Wisniewski, Lavori, Trivedi, Sackeim, Thase, Nierenberg, Quitkin, Kashner, Kupfer, Rosenbaum, Alpert, Stewart, McGrath, Biggs, Shores-Wilson, Lebowitz, Ritz and Niederehe2004; Uher et al. Reference Uher, Maier, Hauser, Marusic, Schmael, Mors, Henigsberg, Souery, Placentino, Rietschel, Zobel, Dmitrzak-Weglarz, Petrovic, Jorgensen, Kalember, Giovannini, Barreto, Elkin, Landau, Farmer, Aitchison and McGuffin2009) will influence guidelines. We applied the most frequently used inclusion and exclusion criteria for classical AETs to a large consecutive series of patients. Comprehensive data on patients' characteristics were available through the extensive routine outcome monitoring (ROM) system. In addition, we investigated whether eligible patients differ from non-eligible patients in treatment outcome.
Method
The Dutch mental healthcare system and treatment steps for major depression
In The Netherlands, health insurance is obligatory for all citizens and regulated by the government. Mental healthcare is easily accessible and not restricted by the financial means of individual patients. The Dutch mental healthcare system is organized in a stepped-care manner and uses evidence-based treatment guidelines. Patients with mood complaints first visit their general practitioner (GP). The treatment guidelines recommended that patients with mild to moderate depression should be treated with psychotherapy or pharmacotherapy, based on the patient's preferences (National Taskforce Guideline, Reference Anderson, Pilling, Barnes, Bayliss and Bird2005). Patients with severe depression should preferably start pharmacotherapy. Rating of severity is based on clinical judgement. Reasons to refer patients to a regional mental healthcare provider (RMPH) are a preference of patients for psychotherapy (not provided by GPs), more severe, recurrent or refractory depression or the presence of co-morbid psychiatric or somatic disorders. After baseline assessment and a clinical interview at our RMPH, patients were offered treatment steps as recommended by the guidelines. Patients suffering from moderate to severe major depression could choose between psychotherapy and antidepressants. For severe depression, antidepressants were the first choice. When patients were already on antidepressants, the dose was optimized or patients were offered the opportunity to switch to another antidepressant or start psychotherapy.
ROM
In 2002, the RMHP Rivierduinen (service area with 1.1 million inhabitants), in collaboration with the University Medical Hospital Leiden, implemented ROM and evidence-based, stepped-care protocols. In ROM, all patients referred to the RMPH for treatment of a mood, anxiety or somatoform disorder have an extensive baseline assessment. Treatment progress is then assessed at 3–4 monthly intervals and before starting a new treatment step. The baseline assessment comprises a standardized diagnostic interview [Mini-International Neuropsychiatric Interview Plus (MINIplus); Sheehan et al. Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller, Hergueta, Baker and Dunbar1998], the collection of sociodemographic and socio-economic data, the administration of disease-specific severity scales and general measures of health. For a more extensive description of ROM, see the design paper of de Beurs et al. (Reference de Beurs, den Hollander-Gijsman, van Rood, van der Wee, Giltay, van Noorden, van der Lem, van Fenema and Zitman2010); see also Supplementary online material.
Patients
To examine the eligibility of depressive out-patients for AETS, we included all out-patients with a DSM-IV diagnosis of a current major depressive disorder as established by MINIplus (Sheehan et al. Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller, Hergueta, Baker and Dunbar1998), who sought treatment at the RMPH Rivierduinen from January 2002 until January 2007. MINIplus does not yield a hierarchy in primary disorder and co-morbid disorders. We included all patients with a major depressive disorder, regardless of the fact whether depression was the primary diagnosis as determined by the treating clinician or a so-called co-morbid disorder. We decided this since primary clinical diagnosis is a concept often used in clinical practice but not well defined in literature and depends heavily on the personal expertise of the individual clinicians. Including patients based on primary clinical diagnosis of depressive disorder only would have led to selection bias and results in a less well-defined sample. Since the presence of a primary clinical diagnosis of depression might influence treatment outcome, we controlled for it in the analyses on the influence of eligibility on treatment outcome.
In order to examine the influence of eligibility to AETs on treatment outcome, we selected all patients in our sample with at least one follow-up assessment in ROM (follow-up group). The treatment outcome of the first treatment step was explored in this project. We examined possible selection bias by comparing the patient characteristics of the follow-up group and the lost-to-follow-up group. We conducted an extensive chart review in the follow-up group in order to obtain information on primary clinical diagnosis and treatment modality. In order to allow comparison with classical AETs, we defined treatment outcome in the same dichotomous variables used in AETs:
(1) Proportion of responders: 50% reduction of the baseline score on the Montgomery–Asberg Depression Rating Scale (MADRS; Asberg et al. Reference Asberg, Montgomery, Perris, Schalling and Sedvall1978).
(2) Proportion of remitters: MADRS⩽10.
Commonly used exclusion criteria for AETs
In an extensive review of the literature, Zimmerman and co-workers identified exclusion criteria that were consistently used in AETs published between 1994 and 1999 in the top five impact ranking journals in the US (Posternak et al. Reference Posternak, Zimmerman, Keitner and Miller2002; Zimmerman et al. Reference Zimmerman, Chelminski and Posternak2004a). We expanded this search by inclusion of AETs published between 1994 and 2007, not only in the aforementioned journals, but also in the remaining journals of the top 10 impact ranking psychiatric journals of 2005. With our expanded search, we obtained 17 additional articles on AETs (Fabre et al. Reference Fabre, Abuzzahab, Amin, Claghorn, Mendels, Petrie, Dube and Small1995; DeMartinis et al. Reference DeMartinis, Schweizer and Rickels1996; Stahl, Reference Stahl2000; Dinan, Reference Dinan2001; Detke et al. Reference Detke, Lu, Goldstein, Hayes and Demitrack2002; Golden et al. Reference Golden, Nemeroff, McSorley, Pitts and Dube2002; Goldstein et al. Reference Goldstein, Mallinckrodt, Lu and Demitrack2002, Reference Goldstein, Lu, Detke, Wiltse, Mallinckrodt and Demitrack2004; Bielski et al. Reference Bielski, Ventura and Chang2004; Trivedi et al. Reference Trivedi, Pigotti, Perera, Dillingham, Carfagno and Pitts2004; Derubeis et al. Reference Derubeis, Hollon, Amsterdam, Shelton, Young, Salomon, O'Reardon, Lovett, Gladis, Brown and Gallop2005; Fava et al. Reference Fava, Ruini and Rafanelli2005; Sir et al. Reference Sir, D'Souza, Uguz, George, Vahip, Hopwood, Martin, Lam and Burt2005; Benkert et al. Reference Benkert, Szegedi, Philipp, Kohnen, Heinrich, Heukels, van der Vegte-Senden, Baker, Simmons and Schutte2006; Langworth et al. Reference Langworth, Bodlund and Agren2006; Mulder et al. Reference Mulder, Joyce, Frampton, Luty and Sullivan2006; Shelton et al. Reference Shelton, Haman, Rapaport, Kiev, Smith, Hirschfeld, Lydiard, Zajecka and Dunner2006). No additional exclusion criteria for AETs were identified. The commonly used exclusion criteria, identified by Zimmerman et al. (Reference Zimmerman, Chelminski and Posternak2004a) are listed below, together with the operationalizations for our sample.
(1) History of DSM-IV manic or hypomanic episodes. At least one (hypo)manic episode on the MINIplus.
(2) Experiencing psychotic features during the current episode of depression. Diagnosis of a current depression with psychotic features on the MINIplus.
(3) Significant risk of suicide. In our sample, suicidality was assessed with the corresponding item on MADRS item 8. Patients with a score of ⩾3 were considered to meet this exclusion criterion. The item is a Likert-scale from 0 to 6: (0, enjoys life, takes it as it is; 2, tired of life, only transient suicidal thoughts; 4, probably better off dead. Suicidal thoughts often occur and suicide is considered to be a possible solution. No specific plans; 6, explicit plans to commit suicide. Active preparation of suicide).
(4) Alcohol or drug abuse or dependency within the last 6 months. Diagnosis of current abuse or dependence on drugs or alcohol on the MINIplus.
(5) Mild depression, as determined by a low baseline score on the Hamilton Depression Scale (HAMD). The most commonly used threshold for inclusion in an efficacy trial is a minimum score of 18 (HAMD 17 items) or a minimum score of 20 (HAMD 21 items) on the HAMD (Hamilton, Reference Hamilton1967; Zimmerman et al. Reference Zimmerman, Posternak and Chelminski2002b). Because in our setting the MADRS is used to assess depression severity, an equivalent of the HAMD score was computed using three previously developed regression equations based on three trials that compared the MADRS and the HAMD17 in out-patients: Mittmann et al. (Reference Mittmann, Mitter, Borden, Herrmann, Naranjo and Shear1997) (A); Hawley et al. (Reference Hawley, Gale, Smith and Sen1998) (B); Zimmerman et al. (Reference Zimmerman, Posternak and Chelminski2004b) (C). Since the Item Response Theory (IRT) has recently been proven to be a probably more reliable method of conversion of the MADRS into the HAMD17 (Carmody et al. Reference Carmody, Rush, Bernstein, Warden, Brannan, Burnham, Woo and Trivedi2006; Uher et al. Reference Uher, Farmer, Maier, Rietschel, Hauser, Marusic, Mors, Elkin, Williamson, Schmael, Henigsberg, Perez, Mendlewicz, Janzing, Zobel, Skibinska, Kozel, Stamp, Bajs, Placentino, Barreto, McGuffin and Aitchison2008), we also used the IRT method to compute proportions of patients not meeting the criterion of minimum baseline severity.
1. (A)
2. (B)
3. (C)
(6) Presence of underlying dysthymic disorder. Diagnosis of dysthymic disorder on the MINIplus.
(7) Illness duration of <4 weeks or >2 years. Duration of <4 weeks or >2 years of the current episode is an exclusion criterion for AETs. Unfortunately, in our sample no reliable information on the duration of the current episode of major depression was available. Therefore, we could not use this exclusion criterion in our analysis.
(8) Presence of co-morbid, non-depressive, non-substance use Axis I disorders. Diagnosis of anxiety disorder, somatoform disorder, eating disorder or attention deficit hyperactivity disorder on the MINIplus.
(9) Presence of borderline personality disorder. In our setting, the Dimensional Assessment of Personality Pathology, short Dutch version (DAPP-SF; Livesley, Reference Livesley, Jackson and Schroeder2006; de Beurs et al. Reference de Beurs, den Hollander-Gijsman, van Rood, van der Wee, Giltay, van Noorden, van der Lem, van Fenema and Zitman2010) was used as a screening instrument for personality pathology. Stringent and less stringent cut-off scores were used to identify patients with a possible personality disorder within a population suffering from mood, anxiety and somatoform disorders (van Kampen et al. Reference van Kampen, de Beurs and Andrea2008). Quartiles (low score–intermediate low–intermediate high–high score) were computed for the patients in our sample on (weighted) scores for the dimensions emotional dysregulation, dissocial behaviour and inhibition. The scores were weighted by the factor loadings derived from research on psychometrics of the DAPP-SF (van Kampen et al. Reference van Kampen, de Beurs and Andrea2008). In our sample, patients with a cut-off of 3.7 and a ‘high score’ on all three dimensions were considered to meet the exclusion criterion of borderline personality according to ‘stringent criteria’. Patients with a cut-off of 2.6 and a ‘high score’ on all three dimensions were considered to meet the exclusion criterion of borderline personality disorder according to ‘less stringent criteria’.
Statistical analysis
For each exclusion criterion, we determined the percentage of patients who met the criterion. For the DAPP-SF, quartiles were computed for (weighted) scores. The scores were weighted by the factor loadings derived from research on psychometrics of the DAPP-SF (de Beurs et al. Reference de Beurs, Rinne, van Kampen, Verheul and Andrea2009). In our sample, there were missing values for the MADRS (n=103) and the DAPP-SF (n=415). Comparison of complete cases and cases with missing data showed differences on many variables. Therefore, it is likely that the missing data were not missing completely at random. Complete case analysis is likely to yield biased estimates (Donders et al. Reference Donders, van der Heijden, Stijnen and Moons2006). Therefore, the multivariate imputation by chained equations (MICE; Royston, Reference Royston2005) method was used to estimate missing values for MADRS. With these imputed data, we computed the percentage of patients meeting the exclusion criteria of mild depression and significant risk of suicide. We did not impute missing values for the DAPP-SF, as this instrument consists of dimensional components that we considered too complex to predict. If the score for the DAPP-SF was missing for a patient, we considered the patient as not meeting the exclusion criterion of presence of borderline personality disorder. Comparison of proportion of responders and remitters in the eligible and non-eligible patient groups were performed by χ2 tests. The influence of the exclusion criteria and ‘eligibility for RCTs’ on treatment outcome was computed by logistic regression after MICE. Odds ratios (OR) and their 95% confidence intervals (CI) were computed by using the robust standard error. Statistical analyses were performed with SPSS v. 16.0 (SPSS Inc., USA) and Stata v. 10.0 (StataCorp, USA).
Results
Patients
A total of 4157 out-patients were assessed at baseline between January 2002 and January 2007. Of these patients, 1653 suffered from a current major depressive disorder according to MINIplus. The demographic features of the 1653 patients are described in Table 1.
Demographics
MADRS, Montgomery–Asberg Depression Rating Scale.
Application of commonly used exclusion criteria for AETs
Bipolarity and psychotic features
A total of 25 of the 1653 patients (1.5%) had at least one (hypo)manic episode (current or history) and 31 patients suffered from a depression with psychotic features (1.9%). There was no overlap between these two groups. Following the approach by Zimmerman et al. (Reference Zimmerman, Mattia and Posternak2002a, Reference Zimmerman, Chelminski and Posternak2004a), we excluded these 56 patients (3.4%) from further analysis. The other exclusion criteria were examined on the remaining 1597 patients.
Suicidality
Of the 1597 patients, 241 patients (15.1%) would have been excluded from AETs because of suicide risk.
Alcohol or drug abuse/dependence
A total of 142 of the 1597 patients (8.9%) met the exclusion criterion of current abuse/dependence on drugs or alcohol.
Severity of the depression at baseline
According to the first regression equation (A), 435 of the 1597 patients (27.2%) did not meet the cut-off score of 18 on the HAMD17. The second and the third regression (B, C) equations yielded identical scores and 664 of the 1597 patients (41.6%) had a score <18 on the HAMD17. The IRT yielded almost identical proportions: 38.7% (cut-off MADRS 24) −44.5% (cut-off MADRS 25) of the patients had a score <18 on the HAMD17.
Co-morbid dysthymic disorder
Altogether, 136 of the 1597 patients (8.5%) met the exclusion criterion for a co-morbid dysthymic disorder.
Other co-morbid Axis I disorders
A total of 1003 of the 1597 patients (62.8%) had co-morbid diagnoses on Axis I according to the MINIplus, 730 patients (45.7%) had at least one anxiety disorder, 180 patients (11.3%) had at least one somatoform disorder. Another 32 patients (2.0%) had other co-morbid disorders.
Personality pathology
In total, 31.6–61.6% of the 1597 patients in our sample may have had some form of personality pathology according to the DAPP-SF. Within this group, the estimated percentage of patients suffering from a borderline personality disorder ranges from three patients (0.2%, stringent criteria) to 112 patients (7.0%, less stringent criteria).
Percentage of patients eligible for AETs and comparison with previous research
Finally, the sample of 1653 depressed out-patients was filtered by stepwise application of the exclusion criteria. Only 17.0–24.5% of our patients would have been eligible for AETs. Stepwise application of the exclusion criteria is described in Fig. 1. Comparison of the incidence of the individual exclusion criteria in our sample with previous research (Zimmerman et al. Reference Zimmerman, Chelminski and Posternak2004a) is described in Table 2.
Stepwise application of commonly used exclusion criteria and the resulting percentages of patients eligible for antidepressant efficacy trials. * Severity at baseline was assessed with the Montgomery–Asberg Rating Scale for Depression. Equivalent Hamilton Rating Scale for Depression scores (17 items version) were calculated using three previously developed regression equations.
Comparison of incidences of exclusion criteria as patient features (%) between our routine outcome monitoring sample and Zimmerman et al.'s (Reference Zimmerman, Chelminski and Posternak2004) sample
Follow-up group
From the 1653 patients suffering from major depression, 46% (n=774) had a follow-up assessment. An extensive chart review was done for those 774 patients. A total of 148 patients had to be excluded from further follow-up analysis due to suspected bipolarity/psychotic features, admission to an in-patient clinic during follow-up, remission on the MADRS at baseline or a time-span between baseline and follow-up assessment, which we considered either to be too short or too long to provide reliable information. Finally, 626 patients were selected for follow-up analysis. In 4% of the 626 patients, information on primary clinical diagnosis was missing. Patient selection is described in Fig. 2.
Selection of the follow-up group. MADRS, Montgomery–Asberg Depression Rating Scale.
In the chart review, we identified that 54% of the selected patients in the follow-up group received antidepressants, either as a solo treatment or in combination with other treatment modalities. Five treatment modalities were identified: ‘antidepressants’ (13%); ‘individual psychotherapy’ (27%, mostly cognitive behavioural therapy or interpersonal therapy); ‘combination of antidepressants and individual psychotherapy (AD+IP)’ (27%); ‘antidepressants and social supportive therapy (AD+SST)’ (14%); ‘other treatment/insufficient information’ (19%). The mean time-span between start of treatment and follow-up assessment was as follows: antidepressant 20.8 weeks (CI 18.7–22.9); individual psychotherapy 20.1 weeks (CI 18.5–21.6); AD+IP 21.5 weeks (CI 20.0–23.1); AD+SST 21.6 weeks (CI 19.9–23.3); other 19.1 weeks (CI 17.1–21.1). In 113 patients, treatment was primarily started for a clinical diagnosis other than major depression, of whom 88 patients received psychotherapeutic treatment focused specifically on anxiety/somatoform disorders.
Lost-to-follow-up analysis
The follow-up group and the lost-to-follow-up group did not differ on most exclusion criteria. The follow-up group only differed from the lost-to-follow-up group in larger proportions of patients with a generalized anxiety disorder [7.3% v. 4.6%, χ2=5.08, degrees of freedom (df)=1, p=0.02] and depression with psychotic features (0.9% v. 2.7%, χ2=7.42, df=1, p=0.01). Based on these results, selection bias was considered to be very small.
Influence of eligibility on treatment outcome
In the follow-up group, 28% of the patients met the criteria for response and 21% of the patients met the criteria for remission. There were no significant differences in response percentages between the patients who would have been eligible for AETs (25%) and those who were not (28%), χ2=0.26, df=1, p=0.61. Remission percentages did not differ either: 16% (eligible patients) v. 23% (non-eligible patients), χ2=1.80, df=1, p=0.18. The influence of patient features commonly used as exclusion criteria on response and remission was examined in multivariate regression models. The following variables were entered as covariates in a multivariate regression model: risk of suicide; minimum baseline severity of depression; co-morbid substance dependency/abuse; co-morbid dysthymia, co-morbid anxiety disorder; co-morbid somatoform disorder; other co-morbid Axis I disorders. ‘Primary clinical diagnosis’ and ‘treatment modality’ were entered in the model as possible confounders. Overall, the explained variance (R 2) was very low for remission (4.1%) and response (1.4%). Only ‘the criterion of minimum baseline severity’ contributed to remission (OR 2.0, CI 1.3–3.1). None of the exclusion criteria contributed significantly to response. The influence of ‘eligibility for AETs’, which we defined as ‘not meeting any of the exclusion criteria’, was investigated in a separate model and did not contribute significantly to response (OR 0.90, CI 0.5–1.8) or remission (OR 1.0, CI 0.5–2.0).
Discussion
We evaluated the eligibility for inclusion in AETs in 1653 out-patients with a major depressive disorder in a Dutch general psychiatric out-patient setting. We followed a model developed for the consistency of exclusion criteria used in AETs (Posternak et al. Reference Posternak, Zimmerman, Keitner and Miller2002; Zimmerman et al. Reference Zimmerman, Chelminski and Posternak2004a). We found that the majority of patients in our sample (75%) did not meet the inclusion criteria. The most common criteria for inclusion that would not have been met were ‘minimum baseline severity of 18 on the HAMD and ‘no co-morbid Axis I disorder’. In addition, we examined the influence of eligibility on treatment outcome. The influence of exclusion criteria on response and remission appears to be small. Only the exclusion of mild depression contributed to improvement of treatment outcome in our sample. Exclusion of less severely depressed patients from the analyses resulted in better treatment outcome. Milder depression is highly prevalent in daily practice and more research into treatment effectiveness in milder depression is warranted.
Comparison with previous research: eligibility for AETs
Our findings are in line with those of previous research (Partonen et al. Reference Partonen, Sihvo and Lonnqvist1996; Zimmerman et al. Reference Zimmerman, Mattia and Posternak2002a; Zetin & Hoepner, Reference Zetin and Hoepner2007; Wisniewski et al. Reference Wisniewski, Rush, Nierenberg, Gaynes, Warden, Luther, McGrath, Lavori, Thase, Fava and Trivedi2009). The percentage of patients eligible for participation in AETs in our study was higher than in earlier research but similar to the latest report on eligibility in the STAR*D trial (Wisniewski et al. Reference Wisniewski, Rush, Nierenberg, Gaynes, Warden, Luther, McGrath, Lavori, Thase, Fava and Trivedi2009). An explanation for the larger proportion of eligible patients in our sample might be the fact that the percentage of patients meeting the criterion of minimum baseline severity was larger in our sample. This might be due to the way in which the Dutch healthcare system is organized. First, there is no (financial) limited access to mental healthcare. Poor socio-economic status has been shown to be associated with more severe pathology and co-morbidity (Lesser et al. Reference Lesser, Leuchter, Trivedi, Davis, Wisniewski, Balasubramani, Petersen, Stegman and Rush2005). Therefore, we expected a higher percentage of patients with more severe depression in our sample. We also expected higher percentages of co-morbid Axis I disorders, but the prevalence of co-morbidity was similar to previous research. Another explanation for the higher percentage of patients who met the criteria for baseline severity is the role of the GP as ‘gatekeeper’ in Dutch healthcare. Nonetheless, a considerable proportion of our patients did not meet the criteria for minimum baseline severity (27–42%). We found lower percentages of bipolarity/psychotic features and borderline personality disorders in our sample. In our RMPH, those patients are often directly (preceding ROM baseline assessment) referred to specialized teams, which might explain the low prevalence in our sample.
Comparison with previous research: influence of eligibility on treatment outcome
In contrast to the recent STAR*D report (Wisniewski et al. Reference Wisniewski, Rush, Nierenberg, Gaynes, Warden, Luther, McGrath, Lavori, Thase, Fava and Trivedi2009), we found no differences in treatment outcome between eligible and non-eligible patients. Together with the marginal explained variance that we found in our model, this suggests that other patient features are more associated with treatment outcome than eligibility for AETs. Many patients, either eligible or not, would not be willing or able to participate in AETs. Participants might differ from non-participants in sociodemographic/socio-economic status, motivation/adherence to treatment and the interaction between clinician and patients. This might also partially explain the differences between our results and those found in the STAR*D report (Wisniewski et al. Reference Wisniewski, Rush, Nierenberg, Gaynes, Warden, Luther, McGrath, Lavori, Thase, Fava and Trivedi2009). In the STAR*D trial, much effort has been undertaken to improve adherence to treatment and to motivate the participating patients and clinicians (Rush et al. Reference Rush, Fava, Wisniewski, Lavori, Trivedi, Sackeim, Thase, Nierenberg, Quitkin, Kashner, Kupfer, Rosenbaum, Alpert, Stewart, McGrath, Biggs, Shores-Wilson, Lebowitz, Ritz and Niederehe2004). It is possible that by ‘controlling’ for these aspects an association between eligibility and treatment outcome can be detected. Unfortunately, the magnitude of the influence of eligibility on treatment outcome is not reported in the STAR*D report and therefore not available for comparison.
Treatment outcome in our study was less favourable than the outcomes typically found in classical AETs and also less favourable than the outcome in STAR*D. A thorough comparison and exploration of differences between the outcomes in RCTs, in more pragmatic trials such as STAR*D and in our ROM project, will be important for daily practice. We are currently performing such a comparison and exploration.
Strengths
The use of ROM in daily mental healthcare provided comprehensive data on a large number of patients. As the only restriction for participation is sufficient language competence and ability to complete computerized or written questionnaires, the results of this type of data collection are very representative of and generalizable to ‘real-life’ psychiatric practice. Furthermore, we consider the fact that the Dutch healthcare system provides unrestricted access to mental healthcare as a strong quality of this research. It diminishes the possibility of selection bias even further.
Limitations
There was a considerable loss to follow-up in our study. In 22% of the lost to follow-up, patients dropped out of treatment directly after baseline assessment. In 38% of the lost to follow-up, patients stayed in treatment, but we had no information on their treatment course. The major reasons for drop-out are unclear; patients might have recovered, were perhaps unsatisfied with the offered treatment or treatment results or had poor compliance. As 38% of the lost-to-follow-up patients remained in treatment, loss to follow-up may also have resulted from factors hampering the ROM follow-up assessments, such as administrative issues or a reduced adherence of clinicians to the ROM methodology. A large loss to follow-up might be a problem in all studies with a more naturalistic design. For example, STAR*D had reached a loss to follow-up of 48% in step II of the study. Of the 4790 patients who were screened at baseline, 12% were not willing to participate, 3% did not meet inclusion criteria, 8% had an HAMD <14 or no data on the HAMD and 25% left the study (Rush et al. Reference Rush, Fava, Wisniewski, Lavori, Trivedi, Sackeim, Thase, Nierenberg, Quitkin, Kashner, Kupfer, Rosenbaum, Alpert, Stewart, McGrath, Biggs, Shores-Wilson, Lebowitz, Ritz and Niederehe2004). Although we had a considerable loss to follow-up, the follow-up group was very similar to the lost-to-follow-up group with regard to criteria for eligibility. We therefore expect the influence of the loss to follow-up on our results to be small.
The absence of information on illness duration is another limitation of this study. Although we expected not to have included patients with an illness duration <4 weeks, as most patients are seen several times by their GP before referral, it is, however, possible that patients were depressed for >2 years. This might have led to an overestimation of the amount of eligible patients. Furthermore, a possible suboptimal diagnostic assessment of borderline personality disorder and the fact that we had no information on physical health problems (not included in Zimmerman's model on exclusion criteria, but still often used as an exclusion criterion), might also have led to an overestimation of the amount of eligible patients. On the other hand, there might be some underestimation of the eligibility in our sample, due to the fact that no data were available on patients who were too ill to complete questionnaires. Not all the patients in our sample were treated with antidepressants. A considerable proportion received other treatment (i.e. psychotherapy) for their depression. However, the percentage of eligible patients turned out to be equal in the antidepressant group and the other-treatment group. For comparability with former research, we used the model of Zimmerman et al., which does not take differences between AETs, such as active versus placebo controlled, into account. Differences in AET architecture will probably influence eligibility, but were not investigated in the present study. Finally, to optimize comparability in treatment outcome with classical RCTs, we used the same definitions of outcome as RCTs, response and remission, determined by a cut-off score. This dichotomization of scales might lead to loss of information compared with continuous outcomes (Uher et al. Reference Uher, Maier, Hauser, Marusic, Schmael, Mors, Henigsberg, Souery, Placentino, Rietschel, Zobel, Dmitrzak-Weglarz, Petrovic, Jorgensen, Kalember, Giovannini, Barreto, Elkin, Landau, Farmer, Aitchison and McGuffin2009).
Note
Supplementary material accompanies this paper on the Journal's website (http://journals.cambridge.org/psm).
Acknowledgements
Ms Wang, Ms Minkenberg, Ms Smalen and Ms Bekke helped in the conductance of the chart review. Mrs R. van der Lem (first author) has received a research grant from ZonMW (grant number 100–002–026), an independent research fund from the Dutch Government.
Declaration of Interest
None.
