Introduction
Mental health disorders place a psychological burden on sufferers, and constitute a large economic burden for society, due to their prevalence, chronicity, association with productivity loss, sick leave and increased healthcare utilization (Wittchen et al. Reference Wittchen, Fuetsch, Sonntag, Müller and Liebowitz2000; Kessler et al. Reference Kessler, Berglund, Demler, Jin, Merikangas and Walters2005). The indirect and direct costs of depression alone have been estimated at $50 billion and $26 billion, respectively, in the USA in 2000 (Wade & Häring, Reference Wade and Häring2010). Healthcare resources are limited and will likely be further constrained as demand and costs grow (Karanikolos et al. Reference Karanikolos, Mladovsky, Cylus, Thomson, Basu, Stuckler, Mackenback and McKee2013). In light of these challenges, healthcare programmes need to place more emphasis on ensuring cost-effectiveness, alongside therapeutic effectiveness for mental health concerns. Cost-effectiveness analysis (CEA) is a tool for investigating the net gains in relation to the incremental costs of a given treatment compared to an alternative (Saha et al. Reference Saha, Hoerger, Pignone, Teutsch, Helfand and Mandelblatt2001).
Internet interventions have demonstrated effectiveness for depression (Richards & Richardson, Reference Richards and Richardson2012) and harmful alcohol use (Riper et al. Reference Riper, Blankers, Hadiwijaya, Cunningham, Clarke, Wiers, Eber and Cuijpers2014), while there is accumulating evidence for interventions targeting anxiety (Arnberg et al. Reference Arnberg, Linton, Hultcrantz, Heintz and Jonsson2014), sleep disturbance (Ritterband et al. Reference Ritterband, Thorndike, Gonder-Frederick, Magee, Bailey, Saylor and Morin2009), smoking cessation (Civljak et al. Reference Civljak, Stead, Hartmann-Boyce, Sheikh and Car2014) and suicidal ideation (Van Spijker et al. Reference Van Spijker, Majo, Smit, van Straten and Kerkhof2014). These interventions are likely to reduce health service delivery costs compared to conventional face-to-face therapy, as they generally involve minimal or no contact with mental health professionals. Internet interventions are therefore assumed to be cost-effective, but it is unclear how strong this evidence is, and what the quality of this evidence is. To answer these questions, systematic reviews are needed.
Previous reviews examining economic evaluations of Internet interventions have focused solely on physical illnesses (Tate et al. Reference Tate, Finkelstein, Khavjou and Gustafson2009), mood and anxiety disorders (Arnberg et al. Reference Arnberg, Linton, Hultcrantz, Heintz and Jonsson2014) or Internet interventions based on cognitive behaviour therapy (CBT) (Hedman et al. Reference Hedman, Ljótsson and Lindefors2012b ). Therefore we aimed to (1) systematically review the available literature on economic evaluations of evidence-based Internet interventions for mental health symptoms or disorders (depression, anxiety, severe health anxiety, harmful alcohol use, smoking cessation, sleep disorders, suicidal ideation); and (2) to review the quality of economic evaluations of Internet interventions.
Method
Search strategy and study selection
A comprehensive literature search of bibliographical databases [PubMed including Medline, EMBASE, the Cochrane Central Register of Controlled Trials, PsycINFO, NHS Economic Evaluations Database (NHS EED), NHS Health Technology Assessment (NHS HTA) Database, and the Office of Health Economics Health Economic Evaluations Database (OHE HEED), Compendex and Inspec] was conducted for relevant articles published between 1990 to 31 July 2014. Terms indicative of Internet-based economic evaluations and mental health disorders were used, with the search limited to ‘humans’, ‘English’, and ‘peer-reviewed journals’ (see Appendix 1 for the full search string).
The identified titles and abstracts were screened for eligibility by two independent researchers (T.D., K.P. or M.B.). Full text copies of all potentially relevant papers, or papers where the abstract provided insufficient detail to determine eligibility, were obtained, screened, and discarded from further analyses if they met exclusion criteria. References of earlier reviews, and reference lists of the included primary articles, were also examined. Data extraction of relevant articles was completed by two independent researchers (T.D. and K.P. or M.B.), with any disagreements resolved through discussion. Randomized controlled trials (RCTs) examining the economic evaluations of Internet-based mental health symptoms or disorders (depression, anxiety, severe health anxiety, harmful alcohol use, smoking cessation, sleep disorders, suicide ideation), compared with a control group, were included. The control group could consist of treatment-as-usual (TAU), another recognized treatment, wait-list or an attention control group. All age groups were included. Only full economic evaluations in which both the cost and consequences of two or more interventions are compared were included in this review. Partial evaluations in which only cost-outcome descriptions were provided were excluded. Studies were excluded if mental health symptoms/disorders were not an outcome, and/or if the focus intervention was not delivered online (e.g. computer-based interventions). Modelling studies were excluded because of methodological differences compared to trial-based economic evaluations (e.g. estimated and synthesized data instead of observational data) which could influence internal validity. Studies were also excluded if the intervention featured only very minimal Internet delivery, or if the intervention targeted a somatic disorder (e.g. irritable bowel syndrome). Conference abstracts, protocol papers, case-studies, non-peer-reviewed papers and non-English papers were also excluded.
Quality assessment
Study quality was assessed with the Drummond 35-item checklist (Drummond & Jefferson, Reference Drummond and Jefferson1996). This tool has been widely used in systematic reviews to assess the quality of economic evaluations (Chen et al. Reference Chen, Madan, Welton, Yahaya, Aveyard, Bauld, Wang, Fry-Smith and Munafò2012; Rodgers et al. Reference Rodgers, Asaria, Walker, McMillan, Lucock, Harden, Palmer and Eastwood2012) and considers a broad range of factors including: the study question; selection of alternatives; form of evaluation; effectiveness data; benefit measurement and valuation of costs and consequences; costing; modelling; adjustments for timing of costs and benefits; allowance for uncertainty; clear presentation of results. Since we have excluded modelling studies, items 20 and 21 regarding ‘modelling’ were not applicable. The study question was rated favourably if authors mentioned hypothesis, research question or objectives/aims. Effectiveness data was rated favourably if the authors provided a brief summary addressing the points in the Drummond guidelines (Drummond & Jefferson, 1996) (selection of study population, method of allocation of subjects, blinding, whether analysed by intention to treat (ITT), effect size with confidence intervals) and a reference to the published source. One author (T.D.) completed the checklist for each study, which was then reviewed by another author (M.B. or B.L.). None of the authors rated papers to which he/she had contributed.
Outcome measures
Outcome measures for the CEA included treatment response (reduction of depression symptoms, anxiety symptoms, alcohol use, smoking behaviour, sleep disorders, suicidal behaviour and self-harm) as assessed with validated mental health scales. The outcome measure for the cost-utility analysis (CUA) was the number of quality-adjusted life years (QALYs) or years lived with disability (YLD) gained as a result of the intervention. Other outcome measures are health-adjusted life expectancies (HALEs) health-adjusted life years (HALYs) or disability-adjusted life years (DALYs), but QALYs are the most common measure used.
Economic evaluation estimates
CEA results are usually summarized in cost-effectiveness ratios (CER), where the costs in the numerator are related to a single common measure of effectiveness in the denominator (e.g. abstinence from alcohol/smoking; Kraemer, Reference Kraemer2008). When comparisons between two interventions are made using this ratio, this is called the incremental cost-effectiveness ratio (ICER). The ICER gives an estimate of the cost for one additional unit of improvement when administering the experimental treatment compared to the control treatment (Bencic et al. Reference Bencic, Bucsics, Dressler, Gerold, Hartinger, Hauser, Huber, Krammer, Kratzer, Muller, Penk, Placheta, Probst, Spanninger, Wieninger, Wild and Windischbauer2006) using the formula
$$\left( {{\rm TC}_{\rm x} \,-\,{\rm TC}_{\rm y}} \right)/({\rm TQ}_{\rm x} \, - \,{\rm TQ}_{\rm y} ),$$
where TCx is the average cost of the experimental intervention, TCy is the average cost of the control intervention, TQx is the proportion of clinically improved participants in the intervention and TQy is the proportion of clinically improved participants in the comparison intervention. Cost-effectiveness can also be presented in terms of cost for 1 year gained living with disability (YLD) averted (Muennig, Reference Muennig2007). Cost-utility ICERs refers to the cost of 1 quality of life year (QALY) gained in the experimental treatment compared to the control condition (Hedman et al. Reference Hedman, Andersson, Ljótsson, Andersson, Rück and Lindefors2011).
In economic evaluations, costs can be determined from several perspectives, including the societal perspective, the third-party payer perspective, the employer perspective and the patient perspective. The perspective taken by the evaluation determines what costs are relevant to, and subsequently included in, the analysis. For example, in the societal perspective (coined the ‘decision-maker approach’ in Drummond et al. Reference Drummond, Sculpher, Torrance, O'Brien and Stoddart2005), health sector costs, other sector costs, patient/family costs and productivity losses are included. In the perspective of the third-party payer perspective, however, only health sector costs are included. For more details, refer to Drummond et al. (Reference Drummond, Sculpher, Torrance, O'Brien and Stoddart2005). The willingness to pay (WTP) gives an indication for the acceptability of the experimental treatment compared to the alternative treatment from a cost-effectiveness point of view by assigning an arbitrary WTP (Muennig, Reference Muennig2007). The principle behind this analysis is that society's WTP for one additional case of improvement determines to which extent a treatment that gives net benefits at higher net costs can be regarded as cost-effective. The threshold of what is considered value for money, which is specified using (among others) QALYs, differs per country. In the UK, for example, the National Institute of Clinical Evidence (NICE) uses a threshold (a WTP) of between £20000 and £30000 per QALY (Mihalopoulos & Chatterton, Reference Mihalopoulos and Chatterton2014). Each economic evaluation study can suffer from several types of uncertainty (e.g. sampling uncertainty). To deal with this type of uncertainty, bootstrap analyses can be conducted. Using bootstrapping techniques with replacement n (often 1000) times a random sample is drawn from the original dataset, resulting in 1000 slightly different samples and thus slightly different ICERs. Of these 1000 ICERs, the percentage can be calculated with (1) more effects and lower costs (dominant); (2) with less effects and lower costs; (3) with more effects and higher costs and (4) with less effects and higher costs (inferior) (Smit et al. Reference Smit, Evers, De Vries and Hoving2013).
Statistical analyses
Where data were available, main outcomes of cost-effectiveness and CUA (CER, ICER, YLD) using ITT analyses at follow-up were reported. ICERs were reported in local currency. In addition, in order to compare ICERs of CUAs across studies, ICERs were converted into pounds Sterling (£) using purchasing power parity exchange rates with 2012 as reference year (the average year of publication of the included studies) (Exchange-rates.org; World Bank Data, 2014). Due to the heterogeneity of the costing methods and the interventions, a formal meta-analysis could not be conducted.
Results
Selection and inclusion of studies
A total of 6602 abstracts were examined (N = 5846 abstracts in total, after removal of duplicates). Potentially eligible full-text papers (N = 236) were retrieved for further consideration, of which 220 were excluded. Sixteen trials met inclusion criteria. There was an excellent inter-rater agreement between the two raters (Cohen's kappa: κ = 0.83). Fig. 1 details a flowchart of the screening process.
PRISMA flowchart.
Characteristics of included studies
A total of 14 031 participants were recruited across 16 studies. Target disorders included depression (n = 4), smoking (n = 3), social phobia (three studies describing two trials), harmful alcohol use (n = 2), panic disorder (n = 1), health anxiety (n = 1), anxiety (n = 1) and suicidal ideation (n = 1). Most studies used CBT as the therapeutic mode of the experimental intervention, and featured support from a coach or therapist. Comparative treatments included group CBT (CBGT), attention-placebo, TAU, unguided Internet intervention, Internet intervention plus telephone support, and Internet-based problem solving therapy (IPST). Five studies included a third comparison intervention, either TAU, wait-list control, attention control, or telephone counselling. The intervention lengths varied between 4 weeks and 6 months.
Clinical effectiveness
All Internet interventions except two (Wallace et al. Reference Wallace, Murray, McCambridge, Khadjesari, White, Thompson, Kalitzaki, Godfrey and Linke2011; Phillips et al. Reference Phillips, Schneider, Molosankwe, Leese, Foroushani, Grime, McCrone, Morriss and Thornicroft2014) demonstrated significant reductions over time in the primary or secondary outcome measures. Similar effects were obtained when Internet interventions were compared to active comparisons (Bergström et al. Reference Bergström, Andersson, Ljótsson, Rück, Andréewitch, Karlsson, Carlbring, Andersson and Lindefors2010; Hedman et al. Reference Hedman, Andersson, Ljótsson, Andersson, Rück and Lindefors2011; Javitz et al. Reference Javitz, Zbikowski, Deprey, Mcafee, Mcclure, Richards, Catz, Jack and Swan2011; Smit et al. Reference Smit, Evers, De Vries and Hoving2013), while two studies found stronger clinical effects for guided v. unguided Internet interventions (Blankers et al. Reference Blankers, Koeter and Schippers2011; Graham et al. Reference Graham, Chang, Fang, Cobb, Tinkelman, Niaura, Abrams and Mandelblatt2012). Four interventions showed significant symptom reductions when Internet interventions were compared to attention controls or TAU (Warmerdam et al. Reference Warmerdam, van Straten, Twisk, Riper and Cuijpers2008; Hollinghurst et al. Reference Hollinghurst, Peters, Kaur, Wiles, Lewis and Kessler2010; Hedman et al. Reference Hedman, Andersson, Lindefors, Andersson, Rück and Ljótsson2012a ; van Spijker et al. Reference Van Spijker, van Straten and Kerkhof2014). However, two studies did not find a significant reduction in alcohol consumption or depression symptomatology compared to attention placebo, respectively (Wallace et al. Reference Wallace, Murray, McCambridge, Khadjesari, White, Thompson, Kalitzaki, Godfrey and Linke2011; Phillips et al. Reference Phillips, Schneider, Molosankwe, Leese, Foroushani, Grime, McCrone, Morriss and Thornicroft2014). Two studies demonstrated similar effects when comparing Internet interventions with TAU (Gerhards et al. Reference Gerhards, de Graaf, Jacobs, Severens, Huibers, Arntz, Riper, Widdershoven, Metsemakers and Evers2010; Smit et al. Reference Smit, Evers, De Vries and Hoving2013).
Economic evaluations
Of the 16 included studies, ten papers describing nine trials took a societal perspective (Gerhards et al. Reference Gerhards, de Graaf, Jacobs, Severens, Huibers, Arntz, Riper, Widdershoven, Metsemakers and Evers2010; Hollinghurst et al. Reference Hollinghurst, Peters, Kaur, Wiles, Lewis and Kessler2010; Warmerdam et al. Reference Warmerdam, Smit, van Straten, Twisk, Riper and Cuijpers2010; Hedman et al. Reference Hedman, Andersson, Ljótsson, Andersson, Rück and Lindefors2011, Reference Hedman, Andersson, Lindefors, Andersson, Rück and Ljótsson2012a , Reference Hedman, El Alaoui, Lindefors, Andersson, Rück, Ghaderi, Kaldo, Lekander, Andersson and Ljótsson2014; Javitz et al. Reference Javitz, Zbikowski, Deprey, Mcafee, Mcclure, Richards, Catz, Jack and Swan2011; Blankers et al. Reference Blankers, Nabitz, Smit, Koeter and Schippers2012; van Spijker et al. Reference Van Spijker, Majo, Smit, van Straten and Kerkhof2012; Smit et al. Reference Smit, Evers, De Vries and Hoving2013), one study took a third-party payer perspective (Graham et al. Reference Graham, Chang, Fang, Cobb, Tinkelman, Niaura, Abrams and Mandelblatt2012) and one study adopted a healthcare provider perspective (Hollinghurst et al. Reference Hollinghurst, Peters, Kaur, Wiles, Lewis and Kessler2010). In five studies, the perspective was not mentioned, but societal perspectives (Nordgren et al. Reference Nordgren, Hedman, Etienne, Bodin, Kadowaki, Eriksson, Lindkvist, Andersson and Carlbring2014) or health insurance perspectives (Titov et al. Reference Titov, Andrews, Johnston, Schwencke and Choi2009; Bergström et al. Reference Bergström, Andersson, Ljótsson, Rück, Andréewitch, Karlsson, Carlbring, Andersson and Lindefors2010; Wallace et al. Reference Wallace, Murray, McCambridge, Khadjesari, White, Thompson, Kalitzaki, Godfrey and Linke2011) could be inferred. In one study (Phillips et al. Reference Phillips, Schneider, Molosankwe, Leese, Foroushani, Grime, McCrone, Morriss and Thornicroft2014), the perspective could not be deduced.
Table 1 provides an overview of main health economic outcomes of the included studies. Notably, the majority of studies reported both CEA and CUA, while six studies performed only CEA. All CUA studies used QALYs as their primary outcome measure, all assessed with EQ-5D (EuroQol Group, 1990). As expected, the CEA studies expressed outcomes differently, using scores of, among others, the Alcohol Use Disorders Identification Test (AUDIT; Saunders et al. Reference Saunders, Aasland, Babor, de la Fuente and Grant1993) and the Beck Depression Inventory (BDI; Beck et al. Reference Beck, Steer, Ball and Ranieri1996). One CEA used YLD as the primary outcome measure. The time horizon of the included studies varied between 6 weeks to 18 months, with the majority being 6 months in length, and one single 4-year follow-up study (Hedman et al. Reference Hedman, El Alaoui, Lindefors, Andersson, Rück, Ghaderi, Kaldo, Lekander, Andersson and Ljótsson2014).
Economic evaluations of Internet interventions at follow-up (intention-to-treat) a
a Unless stated otherwise.
b See Blankers et al. (Reference Blankers, Nabitz, Smit, Koeter and Schippers2012).
c Authors mention NHS perspective but societal perspective is inferred.
AUD, Australian dollars; AUDIT, Alcohol Use Disorders Identification Test; BDI, Beck Depression Inventory; BDI-II, Beck Depression Inventory II; BSS, Beck Suicide Ideation Scale; CBGT, cognitive behavioural group therapy; CBT+, CBT enhanced with other therapeutic techniques, such as motivational interviewing, behavioural self-control, dialectical behavioural therapy or mindfulness based cognitive therapy; CER, cost-effectiveness ratio; CES-D, Centre for Epidemiological Studies Depression; CORE-OM, outcomes in routine evaluation; EI, enhanced internet; EQ-5D, EuroQol-5; F2F, face-to-face; HAI, Health Anxiety Inventory; ICBT, Internet-based cognitive behavioural therapy; ICER, incremental cost-effectiveness ratio; IPST, Internet-based problem solving therapy; LSAS, Liebowitz Social Anxiety Scale; mo, months; MPP, multiple point prevalence; NR, not reported; PDSS, Panic Disorder Severity Scale; PHQ-9, Patient Health Questionnaire; PTC, proactive telephone counselling; QALY, quality-adjusted life years; RCT, randomized controlled trial; RT, randomized trial; SIAS, Social Interaction Anxiety Scale; SPS, Social Phobia Scale; SPP, single point prevalence; TAU, treatment as usual; TOT-AL, total past week alcohol consumption; WTP, willingness to pay; YLD, years lived with disability.
Depression
From a cost-effectiveness perspective, guided Internet ICBT and guided IPST showed high probabilities of being more cost-effective compared to wait-list (Warmerdam et al. Reference Warmerdam, Smit, van Straten, Twisk, Riper and Cuijpers2010). CUA, however, led to modest results regarding the cost-effectiveness of Internet interventions. With an ICER of £19 371 and £9873 for 1 additional QALY, ICBT and IPST, respectively, had a 50% likelihood of being more acceptable than wait-list. Uncertainty analysis demonstrated that the interventions produced more effects at higher costs compared to wait-list. In one study (Hollinghurst et al. Reference Hollinghurst, Peters, Kaur, Wiles, Lewis and Kessler2010), guided ICBT had better outcomes but at higher costs compared to TAU, with a 50% likelihood of ICBT being more cost-effective in terms of QALYs than TAU at ICER = £19 322 per QALY threshold. For ICERs in local currency, please see Table 1. Unguided ICBT produced similar effectiveness against lower costs compared to TAU (Gerhards et al. Reference Gerhards, de Graaf, Jacobs, Severens, Huibers, Arntz, Riper, Widdershoven, Metsemakers and Evers2010). ICBT had a 65% probability of being cost-effective compared to TAU at a WTP of £0 (Gerhards et al. Reference Gerhards, de Graaf, Jacobs, Severens, Huibers, Arntz, Riper, Widdershoven, Metsemakers and Evers2010). One study examining unguided ICBT on work-related performance and other outcomes, including depression, found no differential effects or cost-effects compared to a psycho-educational control condition (Phillips et al. Reference Phillips, Schneider, Molosankwe, Leese, Foroushani, Grime, McCrone, Morriss and Thornicroft2014).
Anxiety disorders (including health anxiety)
Guided ICBT generated a societal economic gain (being more clinically efficacious at a lower societal cost) for health anxiety (Hedman et al. Reference Hedman, Andersson, Lindefors, Andersson, Rück and Ljótsson2012a ), social anxiety (Hedman et al. Reference Hedman, Andersson, Ljótsson, Andersson, Rück and Lindefors2011), panic disorder (Bergström et al. Reference Bergström, Andersson, Ljótsson, Rück, Andréewitch, Karlsson, Carlbring, Andersson and Lindefors2010) and anxiety disorder in general (Nordgren et al. Reference Nordgren, Hedman, Etienne, Bodin, Kadowaki, Eriksson, Lindkvist, Andersson and Carlbring2014), compared to attention controls or CBGT, at post-test or 3–6 months’ follow-up. However, in a 4-year follow-up study on social anxiety (Hedman et al. Reference Hedman, El Alaoui, Lindefors, Andersson, Rück, Ghaderi, Kaldo, Lekander, Andersson and Ljótsson2014), ICBT and CBGT yielded similar results in terms of cost-effectiveness. In a study targeting social anxiety (Titov et al. Reference Titov, Andrews, Johnston, Schwencke and Choi2009), the cost for 1 year gained living with disability (YLD) averted was lower in ICBT compared to CGBT treatment at follow-up. At WTP $0 for an additional case of improvement, ICBT demonstrated a 64% and 81% probability of being cost-effective compared to attention control and CBGT, respectively (Hedman et al. Reference Hedman, Andersson, Ljótsson, Andersson, Rück and Lindefors2011, Reference Hedman, Andersson, Lindefors, Andersson, Rück and Ljótsson2012a ). However, at 4-year follow-up, the probability of ICBT being cost-effective compared to CGBT for social anxiety diminished from 81% to 62% at WTP £0. In the CUA, guided ICBT for health anxiety had a 67% probability of being cost-effective if society would pay £0 for one gained QALY at post-test or follow-up, whereas social anxiety had a 81% probability, of being cost-effective if society would pay £0 for one gained QALY at post-test. These interventions (Hedman et al. Reference Hedman, Andersson, Ljótsson, Andersson, Rück and Lindefors2011, Reference Hedman, Andersson, Lindefors, Andersson, Rück and Ljótsson2012a , Reference Hedman, El Alaoui, Lindefors, Andersson, Rück, Ghaderi, Kaldo, Lekander, Andersson and Ljótsson2014; Nordgren et al. Reference Nordgren, Hedman, Etienne, Bodin, Kadowaki, Eriksson, Lindkvist, Andersson and Carlbring2014) dominated their controls, leading to better outcomes at lower costs. Each QALY gained in ICBT for health anxiety generated a societal earning of £6688 compared to attention control at post-test. Each QALY gained in ICBT to reduce social anxiety generated a societal earning of £ 11 307 compared to CBGT at 6 months’ follow-up, but cost-effectiveness results diminished to a 50% probability at ICER = £4660 for one additional QALY. For anxiety disorders in general (Nordgren et al. Reference Nordgren, Hedman, Etienne, Bodin, Kadowaki, Eriksson, Lindkvist, Andersson and Carlbring2014), ICBT had a 90% probability of being cost-effective at WTP £0. Each QALY gained in ICBT for anxiety generated a societal earning of £4732 compared to an active wait-list (similar to TAU) at post-test. For ICERs in local currency, please see Table 1.
Alcohol misuse
Guided ICBT+ (CBT enhanced with other therapeutic techniques, such as motivational interviewing or behavioural self-control) compared to unguided ICBT+ to reduce alcohol consumption generated a societal economic cost at follow-up (Blankers et al. Reference Blankers, Nabitz, Smit, Koeter and Schippers2012). Guided ICBT+ led to additional effects and a better QALY health gain at additional costs relative to unguided ICBT+. With ICER = £12 228 for one additional QALY, guided ICBT+ and unguided ICBT+ would be equally preferable. When society is willing to pay more than £12 228, guided ICBT+ would probably be more cost-effective than unguided ICBT+ (Blankers et al. Reference Blankers, Nabitz, Smit, Koeter and Schippers2012). For ICERs in local currency, please see Table 1. One study (Wallace et al. Reference Wallace, Murray, McCambridge, Khadjesari, White, Thompson, Kalitzaki, Godfrey and Linke2011) showed no significant differences in EQ-5D scores when unguided ICBT+ was compared to an attention control condition to reduce alcohol consumption, and therefore no CER was calculated. However, the ICBT intervention in this evaluation costs significantly more than the attention control.
Smoking
Internet-based multiple tailoring was more cost-effective compared to the same intervention provided as face-to-face counselling or usual care, when smoking abstinence was the outcome measure (Smit et al. Reference Smit, Evers, De Vries and Hoving2013). However, when quality of life was used as an outcome measure, multiple tailoring was dominated by usual care because this treatment was both more expensive and less effective. Furthermore, multiple tailoring and counselling was more expensive but also more effective than usual care and multiple tailoring in increasing the QALYs gained. This resulted in an incremental cost of £33 124 per QALY gained when comparing multiple tailoring and counselling with usual care, and in an incremental cost of £15 097 per QALY when comparing multiple tailoring and counselling to multiple tailoring only (Smit et al. Reference Smit, Evers, De Vries and Hoving2013). A telephone-guided smoking cessation Internet intervention was more cost-effective compared to the same Internet intervention without telephone support (Graham et al. Reference Graham, Chang, Fang, Cobb, Tinkelman, Niaura, Abrams and Mandelblatt2012), whereas an Internet-based counselling intervention was more cost-effective than proactive telephone counselling only (Javitz et al. Reference Javitz, Zbikowski, Deprey, Mcafee, Mcclure, Richards, Catz, Jack and Swan2011).
Suicidal ideation
Unguided ICBT+ dominated the attention control at post-test, leading to better outcomes at lower costs for suicide prevention (van Spijker et al. Reference Van Spijker, Majo, Smit, van Straten and Kerkhof2012) With a WTP £0, there was a 93% probability that the Internet intervention would be regarded as more cost-effective than attention control.
Guidance
Guided Internet interventions seem to be cost-effective compared to both group treatment (Titov et al. Reference Titov, Andrews, Johnston, Schwencke and Choi2009; Bergström et al. Reference Bergström, Andersson, Ljótsson, Rück, Andréewitch, Karlsson, Carlbring, Andersson and Lindefors2010; Hedman et al. Reference Hedman, Andersson, Ljótsson, Andersson, Rück and Lindefors2011) and TAU/active control group (Hollinghurst et al. Reference Hollinghurst, Peters, Kaur, Wiles, Lewis and Kessler2010; Nordgren et al. Reference Nordgren, Hedman, Etienne, Bodin, Kadowaki, Eriksson, Lindkvist, Andersson and Carlbring2014), attention control (Hedman et al. Reference Hedman, Andersson, Lindefors, Andersson, Rück and Ljótsson2012a ), telephone counselling (Javitz et al. Reference Javitz, Zbikowski, Deprey, Mcafee, Mcclure, Richards, Catz, Jack and Swan2011), unguided Internet interventions (Blankers et al. Reference Blankers, Nabitz, Smit, Koeter and Schippers2012; Graham et al. Reference Graham, Chang, Fang, Cobb, Tinkelman, Niaura, Abrams and Mandelblatt2012) and wait-list (Warmerdam et al. Reference Warmerdam, Smit, van Straten, Twisk, Riper and Cuijpers2010). However, in the latter study, results from CUA for ICBT and IPST for depression were less robust (Warmerdam et al. Reference Warmerdam, Smit, van Straten, Twisk, Riper and Cuijpers2010). The most expensive ICER from CUA was £33 124 (multiple tailoring and face-to-face counselling for smoking cessation), meaning that at a WTP of ⩾£33 124 per QALY, the Internet interventions are more attractive from the chosen cost-effectiveness perspective than the control conditions
From a cost-effectiveness analysis point of view, three unguided Internet interventions for suicide prevention, depression and smoking cessation demonstrated cost-effectiveness (Gerhards et al. Reference Gerhards, de Graaf, Jacobs, Severens, Huibers, Arntz, Riper, Widdershoven, Metsemakers and Evers2010; van Spijker et al. Reference Van Spijker, Majo, Smit, van Straten and Kerkhof2012; Smit et al. Reference Smit, Evers, De Vries and Hoving2013). However, data derived from CUA results were more modest for depression and smoking cessation compared to TAU; in these studies, usual care would probably be equally or more cost-effective compared to unguided Internet interventions effectiveness (Gerhards et al. Reference Gerhards, de Graaf, Jacobs, Severens, Huibers, Arntz, Riper, Widdershoven, Metsemakers and Evers2010; Smit et al. Reference Smit, Evers, De Vries and Hoving2013). In one study (Wallace et al. Reference Wallace, Murray, McCambridge, Khadjesari, White, Thompson, Kalitzaki, Godfrey and Linke2011), direct costs of unguided ICBT+ for alcohol consumption were higher compared to attention control, while there were no significant differences in effect on outcome or EQ-5D scores. Finally, in another unguided ICBT study, no significant differences in EQ-5D scores were found when unguided ICBT was compared to a psycho-educational attention control condition to reduce symptoms of depression (Phillips et al. Reference Phillips, Schneider, Molosankwe, Leese, Foroushani, Grime, McCrone, Morriss and Thornicroft2014).
Economic perspective
Overall, guided Internet interventions appear to be cost-effective from the perspective of society, healthcare providers and third-party payers (Hollinghurst et al. Reference Hollinghurst, Peters, Kaur, Wiles, Lewis and Kessler2010; Warmerdam et al. Reference Warmerdam, Smit, van Straten, Twisk, Riper and Cuijpers2010; Hedman et al. Reference Hedman, Andersson, Ljótsson, Andersson, Rück and Lindefors2011, Reference Hedman, Andersson, Lindefors, Andersson, Rück and Ljótsson2012a , Reference Hedman, El Alaoui, Lindefors, Andersson, Rück, Ghaderi, Kaldo, Lekander, Andersson and Ljótsson2014; Javitz et al. Reference Javitz, Zbikowski, Deprey, Mcafee, Mcclure, Richards, Catz, Jack and Swan2011; Blankers et al. Reference Blankers, Nabitz, Smit, Koeter and Schippers2012; Graham et al. Reference Graham, Chang, Fang, Cobb, Tinkelman, Niaura, Abrams and Mandelblatt2012). Unguided Internet interventions were cost-effective at WTP £0 from a societal perspective when depression, suicide ideation or smoking abstinence was the outcome measure, but not when QALYs were the outcome measure for depression and smoking abstinence, because there were no differential clinical effects (Gerhards et al. Reference Gerhards, de Graaf, Jacobs, Severens, Huibers, Arntz, Riper, Widdershoven, Metsemakers and Evers2010; van Spijker et al. Reference Van Spijker, Majo, Smit, van Straten and Kerkhof2012; Smit et al. Reference Smit, Evers, De Vries and Hoving2013). Of the studies which did not mention their economic perspective (but inferred societal or health insurance perspectives), guided Internet intervention appeared to be cost-effective (Nordgren et al. Reference Nordgren, Hedman, Etienne, Bodin, Kadowaki, Eriksson, Lindkvist, Andersson and Carlbring2014), but only when direct intervention costs were included (Titov et al. Reference Titov, Andrews, Johnston, Schwencke and Choi2009; Bergström et al. Reference Bergström, Andersson, Ljótsson, Rück, Andréewitch, Karlsson, Carlbring, Andersson and Lindefors2010). However, unguided Internet interventions were not cost-effective (Wallace et al. Reference Wallace, Murray, McCambridge, Khadjesari, White, Thompson, Kalitzaki, Godfrey and Linke2011; Phillips et al. Reference Phillips, Schneider, Molosankwe, Leese, Foroushani, Grime, McCrone, Morriss and Thornicroft2014). Taken together, the economic perspective does not seem to be strongly related to cost-effectiveness outcomes.
Quality
Based on Drummond's checklist (Drummond & Jefferson, Reference Drummond and Jefferson1996), the quality of the included economic evaluations varied (see Table 2). On average, the studies scored 72% (338/471) of the items positive. Four studies (25%) did not mention the economic perspective. Eleven papers provided insufficient information about details of the design and results of the effectiveness study and therefore was rated unclear. The majority of them provided insufficient information on the method of allocation concealment of subjects. One paper was rated unfavourable because of the absence of ITT data. Furthermore, five studies (31%) did not include uncertainty analysis and/or sensitivity analysis. Most studies (N = 12; 75%) were evaluated over a short time frame (6 weeks–6 months). Three studies (19%) reported less than 60% of the necessary details recommended by economic guidelines (Drummond & Jefferson, Reference Drummond and Jefferson1996). Two studies targeting smoking cessation held a societal perspective, but did not include productivity costs and/or healthcare costs (Javitz et al. Reference Javitz, Zbikowski, Deprey, Mcafee, Mcclure, Richards, Catz, Jack and Swan2011; Smit et al. Reference Smit, Evers, De Vries and Hoving2013). Because of the economic evaluation methodology (e.g. variation in economic perspectives, economic evaluations, comparison groups), comparison of results between studies was hampered. For example, alongside differences in included costs according to the chosen perspective, differences in methods for including or excluding costs were also apparent, such as that in some studies, development of intervention costs were included, whereas in other studies these costs were considered as sunk costs. However, ten of the included studies (62.5%) adhered to ⩾75% of the guidelines and therefore achieved a rating of good quality.
Quality of studies investigating cost-effectiveness of Internet-based interventions
n.a., Not applicable.
Discussion
Main findings
The aim of this review was to provide an overview of outcomes and quality of economic evaluations of Internet interventions compared to TAU, CBGT, attention control, telephone counselling or unguided Internet CBT for a range of mental health disorders. Concerning the intervention modality, the most robust evidence for cost-effectiveness was found for guided Internet interventions at a WTP range of £1801–£33 124 per QALY. With regards to the target disorder, the strongest evidence was found for anxiety disorders, followed by depression, smoking cessation and alcohol misuse. Overall, long-term follow-up data revealed higher costs per effect measure. Except for suicide ideation, cost-effectiveness of unguided Internet interventions for depression, alcohol and smoking cessation demonstrated weaker effects. Particularly with CUA, the Internet intervention was more expansive per QALY compared to the control group. This may be due to limited effects in some trials.
Most studies took a societal perspective, which is the broadest possible perspective, including indirect as well as direct costs. Several of the included studies claimed to have employed a societal perspective. However, whether some of these studies really employed a societal perspective is questionable, and seem to rather have a partial societal perspective at best with a largely health sector perspective and the addition of productivity impacts. Interestingly, two studies targeting smoking cessation from a societal perspective did not include productivity costs, but one may argue that these costs are less relevant when smoking cessation is the primary outcome.
Quality of economic evaluations of Internet interventions
The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist (Husereau et al. Reference Husereau, Drummond, Petrou, Carswell, Moher, Greenberg, Augustovski, Briggs, Mauskopf and Loder2013) is a newly developed checklist which aims better reporting of economic evaluations, and ultimately to better health decisions. Although CHEERS is intended to, and may be a very good checklist for reporting economic evaluations, we found the checklist to be difficult to use as a quality rating instrument. For example, some items were not specific enough for quality rating purposes. Therefore, we preferred to use the Drummond 35-item checklist (Drummond & Jefferson, Reference Drummond and Jefferson1996). The 35-item checklist is a widely supported checklist, and is incorporated in the latest version of the Cochrane Handbook for Systematic Reviews of Interventions as one of two instruments to rate the risk of bias in economic evaluations. Topics addressed in both instruments are comparable.
Economic evaluations of interventions rely on the assessment of their clinical effectiveness. Any limitation which weakens the assessment of effectiveness weakens any economic evaluations based on it. The quality of the effectiveness study could be captured in assessing the risk of bias. However, none of the common reporting checklists (CHEERS or Drummond lists) requires authors to report risk of bias (e.g. sequence generation, allocation concealment, blinding) explicitly. The majority of economic evaluations included provided insufficient details on allocation method. However, this information may likely have been provided in the source publication. All included studies used an RCT design, and analysed data by ITT (except one paper). Blinding participants for treatment allocation is rarely achievable in intervention trials for mental health disorders. However, as most studies used online self-report questionnaires, assessors were blind for participant outcomes. The quality of economic evaluations of the included studies varied. Most studies had short time horizons for evaluation, which may yield conservative cost estimates. While some studies lacked a considerable amount of detail, more than half of the included studies demonstrated almost full adherence to the economic guidelines. All of them featured sensitivity and/or uncertainty analyses, which increases the robustness of results. Similar to other economic evaluation reviews (Kraemer, Reference Kraemer2008; Mihalopoulos & Chatterton, Reference Mihalopoulos and Chatterton2014), our review revealed considerable differences in methodology across studies (e.g. economic perspective, economic analysis, comparator intervention, outcome measures, included costs). Several papers included development costs of the intervention in the CEA. As to whether or not to take these costs into account is a matter of debate. Ronckers et al. (Reference Ronckers, Groot and Ament2014), for example, argue that it is important to only include those costs that will have to be incurred if the intervention is performed again. This means that development costs and costs incurred for research purposes should not be included and are considered as ‘sunk costs’. However, particularly with increasing complexity of Internet interventions, development costs can be continuous. Therefore, although these sunk costs are not part of the CEA, it is informative to report the development costs. As such, any ongoing maintenance, content-update or refinement costs should be included in the intervention costs. Furthermore, differences existed for measures of effectiveness and cost per clinical outcome (e.g. measures of reduction in symptoms, abstinence, quitters). These CERs may be useful for direct comparison to other programmes, but cannot be easily compared to outcomes of programmes for other health conditions. Instead, comparable standardized outcome measures, like QALYs are preferred (Mihalopoulos & Chatterton, Reference Mihalopoulos and Chatterton2014). The majority of included studies used QALYs to calculate cost-utility outcomes, thereby increasing the comparability of outcomes of the different Internet interventions. For this purpose, future researchers are advised to include CUA using QALYs in their economic evaluations.
Comparison with prior work
Our finding that guided Internet interventions are cost-effective compared to e.g. group treatment, attention control group, or wait-list echoes earlier reviews of Internet interventions focussing on ICBT (Hedman et al. Reference Hedman, Ljótsson and Lindefors2012b ; Arnberg et al. Reference Arnberg, Linton, Hultcrantz, Heintz and Jonsson2014) and physical illnesses (Tate et al. Reference Tate, Finkelstein, Khavjou and Gustafson2009).
Strengths and limitations
One of the strengths of this systematic review included the comprehensive search strategy used, including a number of economic databases. Another important strength was the use of multiple study assessors achieving high inter-rater agreement. However, some limitations of the current review should be noted. First, due to the variability of methods used in the included studies, the comparison of results was hindered and we were unable to conduct a meta-analysis. Second, the time horizon of the included economic evaluations differed, with two studies using a time horizon of only 6 weeks but the majority spanning 6 months. However, comparing studies with shorter and longer time horizons did not influence the conclusions. Third, by expressing ICERs in comparable currency can also give a false sense of comparability since costs may be collected and valued differently hence reducing comparability. In addition, several studies lacked uncertainty analysis and/or sensitivity analysis. Finally, despite an extensive search, the number of studies was small, and most studies used a wait-list control or attention placebo as a comparison group, instead of the most cost-effective intervention currently available, which restricted our interpretations as to whether Internet interventions are cost-effective.
Implications
The economic evaluation studies included in this review demonstrated that Internet interventions, and guided Internet interventions in particular, compared favourably to, or surpassed the cost-effectiveness of wait-list, attention-placebo and traditional services, including CBGT, unguided CBT and TAU. These initial results are promising and suggest that if access to guided Internet interventions were increased, this could result in significant cost savings and reduced service demand on the health system, whilst improving mental health outcomes and quality of life for patients. However, more research is needed to test this.
Future research
More economic valuations are needed, especially comparing guided Internet interventions and face-to-face interventions or the most cost-effective intervention currently available directly instead of wait-list or attention controls, and economic evaluations for disorders not addressed (e.g. specific anxiety disorders, insomnia). Interpretation of results of economic evaluation of Internet interventions may be significantly improved by increasing comparability between the studies, e.g. by using standardized generic measures, and a greater degree of agreement as to the necessary costs to include in evaluations (especially regarding intervention development costs). Furthermore, longer follow-ups, and increased adherence to economic evaluation guidelines such as the Drummond checklists (including uncertainty and sensitivity analyses) will increase the robustness of results. With the emerging field of eMental health, earlier developed evidence-based self-help manuals (e.g. Bower et al. Reference Bower, Richards and Lovell2001) seem to be forgotten. However, given their potential for cost-effective treatments, it would be of value to investigate this further.
As mental illnesses are associated with profound economic consequences, both to the individual and to wider society (Gilbody et al. Reference Gilbody, Bower and Whitty2006), the societal perspective might be the most ideal perspective of most value to policy makers. Another advantage is that a wide perspective allows narrowing down in secondary analyses. Therefore we advise future researchers to employ the societal perspective.
Conclusions
Guided Internet interventions for depression, anxiety, smoking cessation and alcohol consumption demonstrated higher probabilities of being more cost-effective than controls at an ICER range of £1801–33 124 (the point of indifference). However, the evidence for unguided Internet interventions for depression and smoking cessation was less convincing. Most studies adhered reasonably to economic guidelines. With increasing pressure on healthcare budgets across the globe, strategies to improve access to mental healthcare at lower cost are needed. Results of this review are promising, pointing to the possible inclusion of guided Internet interventions in these strategies, but more economic evaluations for guided Internet interventions compared to the most cost-effective intervention currently available, is needed in order to determine cost-effectiveness of Internet interventions.
Supplementary material
For supplementary material accompanying this paper visit http://dx.doi.org/10.1017/S0033291715001427
Acknowledgements
This study was funded by the Black Dog Institute, University of New South Wales, and VU University Amsterdam. H.C. is supported by National Health and Medical Research Council Fellowship 1 056 964. E.H. and B.L. are supported by Karolinska Institutet and the Stockholm County Council, Stockholm, Sweden, which are public institutions and had no role in the design and conduct of the study; the collection, management, and analysis of the data; or in the preparation, review and approval of the manuscript.
Declaration of Interest
Authors M.B., B.L. and E.H. are authors of some of the included studies.
