Pre-treatment differences

Whilst pre-treatment dissociation and BPD severity were equivalent between treatment groups, DBT patients exhibited significantly greater emotional dysregulation and likelihood of PTSD, self-harming behaviour, and A&E admission in the 12 months before treatment (67% v. 34%) (Barnicot and Crawford, Reference Barnicot and Crawford2019, Table 1). Additionally, DBT patients exhibited clinically significantly, (although statistically non-significantly), higher rates of depression (43% v. 25%), alcohol dependence (25% v. 13%) and psychotic symptoms (46% v. 35%). These may be chance differences, or viewed collectively, may raise questions about referral bias and participant self-selection, reinforcing the importance of randomised treatment allocation in future studies.

Reporting of findings

We agree that pre-registration and specification of primary outcomes is essential for randomised controlled trials. However, our study was not a clinical trial. We have fully reported and discussed all findings in our abstract, results and discussion.

Change in self-harm and emotional dysregulation

We based our conclusion that reductions in self-harm and emotional dysregulation were greater in DBT on our finding that, in both unadjusted and adjusted multi-level mixed-effects models, there was a significant treatment-by-time interaction favouring DBT for these outcomes. Two trials very recently published in this journal (July 2019 issue) used similar models to examine the rate of change in the outcome variables as their primary analysis (Freyer-Adam et al., Reference Freyer-Adam, Baumann, Haberecht, Bischof, Meyer, Rumpf, John and Gaertner2019; Strauss et al., Reference Strauss, Granholm, Holden, Ruiz, Gold, Kelly and Buchanan2019), as did a largescale trial of MBT conducted by the treatment developer (Bateman and Fonagy, Reference Bateman and Fonagy2009). Further post-hoc analysis of our treatment-by-time interaction effect shows a larger decrease in emotional dysregulation and self-harm per month in DBT patients. MBT patients showed an estimated 1.08 point decrease in emotional dysregulation per month (β = −1.08, 95% CI −1.74 to −0.43, p = 0.001) compared to a 2.49 point estimated decrease per month in DBT patients (β = −2.49, 95% CI −3.15 to −1.83, p < 0.001). MBT patients showed an estimated 0.95 times decrease in the rate of self-harm per month (IRR = 0.95, 95% CI 0.91 to 0.99, p = 0.008) and DBT patients a 0.89 times estimated decrease per month (IRR = 0.89, 95% CI 0.86 to 0.92, p < 0.001). This fully justifies our conclusion that reductions in self-harm and emotional dysregulation were greater in DBT.

Luyten and colleagues usefully highlight the lack of between-groups difference in emotional dysregulation and self-harm at the 12-month follow-up (Barnicot and Crawford, Reference Barnicot and Crawford2019, Table 2), suggesting therefore that patients followed different trajectories towards the same endpoint. Unlike our mixed-effects models – which benefited from incorporating data from multiple timepoints and maximum-likelihood-based modelling, increasing statistical power and robusticity to loss to follow-up (Tango, Reference Tango2016) – our sample size for outcome comparison at 12 months was restricted by using data from a single timepoint, and by loss to follow-up. Substantial overdispersion in self-harm incidents further reduced statistical power. We agree that the 6-point difference in emotional dysregulation score at month 12 is neither statistically nor clinically significant – thus whilst the rate of improvement in emotional dysregulation was significantly faster amongst DBT patients, outcomes at the end of 12 months were similar. By contrast, there were large and clinically significant (but statistically non-significant) differences in self-harm. The median number of self-harm incidents between months 10 and 12 was 2 in the DBT condition and 12.5 in the MBT condition (Barnicot and Crawford, Reference Barnicot and Crawford2019, Table 2 – medians are presented due to the skewed data distribution (Field, Reference Field2013)). Thus, one possible interpretation is that the faster rate of improvement in self-harm in DBT was accompanied by a clinically significantly lower rate of self-harm by 12 months, which failed to reach statistical significance. A largescale adequately powered trial is required to determine whether there is a statistically significant difference in self-harm at 12 months or any other timepoint.

Reporting of findings from unadjusted and adjusted models

We strongly refute the suggestion that we have shown allegiance bias by only presenting adjusted models where findings from unadjusted models disadvantaged DBT. Firstly, whilst we are not privy to all unconscious biases that dwell within us, Barnicot is a non-clinical academic unaffiliated with any particular therapeutic paradigm; Crawford is a trained MBT therapist and delivered MBT in one of the participating treatment centres. Thus, any bias might be expected to favour MBT, not DBT. Secondly, our choice of when to present adjusted models was based solely on an objective criterion: if an outcome was found to differ significantly between DBT and MBT patients in an unadjusted model, we presented findings from an adjusted model, in order to test whether the significant difference in the unadjusted model still held after adjusting for potential confounders. If an outcome was not found to differ significantly between DBT and MBT patients in unadjusted models, we did not run an adjusted model (Barnicot and Crawford, Reference Barnicot and Crawford2019, Methods). This criterion applied both to findings favouring DBT and those favouring MBT.

Treatment dropout

Noting the markedly higher treatment dropout rate in DBT, and cautious of attributing any outcome differences to inherent effects of therapy modality that might instead be better explained by treatment dropout or factors linked to treatment dropout, we conservatively chose to adjust our models for any such factors. Therefore, as well as including baseline covariates differing significantly between DBT and MBT patients, we also included baseline covariates differing significantly between treatment completers and dropouts (Barnicot and Crawford, Reference Barnicot and Crawford2019, Methods). Guidance on clinical trial analysis agrees that adjustment for treatment dropout can be useful in sensitivity analyses (European Medicines Agency Committee for Medicinal Products for Human Use, 2015). However, we agree with Luyten and colleagues that caution is merited, as if a higher treatment dropout rate is a true property of DBT treatment and also associated with poorer outcomes, then adjusting for variance shared between DBT treatment and treatment dropout could obscure evidence of poorer outcomes in DBT (European Medicines Agency Committee for Medicinal Products for Human Use, 2015). Our analyses were conducted on an intention-to-treat basis, including data collected from treatment dropouts following their treatment discontinuation. The requested replication of our analyses in the sample of treatment completers alone would restrict our sample size to n = 43 at baseline and n = 40 at month 12, reducing the power of the study to detect any treatment differences. We do not believe this would be useful.

Covariate-adjusted models in non-randomised studies

Luyten and colleagues cite an important difficulty with adjustment for covariates whose effects overlap with treatment effects. This is particularly problematic in non-randomised studies, where it is impossible to determine whether pre-treatment differences between groups have arisen by chance or whether they constitute an inherent property of group membership, and thus by removing variance that is shared between the covariate and treatment exposure, adjustment for group differences may obscure or enhance treatment effects (Miller and Chapman, Reference Miller and Chapman2001; Field, Reference Field2013). However, this problem arises when covariates are related to treatment exposure, not to the dependent variable (Field, Reference Field2013, pg. 484). Thus, the request that we exclude any covariates associated with our dependent variables would not solve the problem of shared variance with the treatment effect and would be contrary to clinical trial guidance (European Medicines Agency Committee for Medicinal Products for Human Use, 2015). Despite this difficulty, UK National Institute for Health and Care Excellence (NICE) Decision Support Unit guidance stresses the importance of evaluating and adjusting for potentially confounding differences between treatment groups in non-randomised studies (Faria et al., Reference Faria, Hernandez Alava, Manca and Wailoo2015). A possible compromise may be found in the NICE recommendation that authors present sensitivity analyses to test whether their findings are robust to different analysis methods (Bell et al., Reference Bell, Wailoo, Hernandez, Grieve, Faria, Gibson and Grimm2016).