In many fields, such as rehabilitation or self-management, multicomponent interventions (MCIs) are common (1). They may be recommended as part of guidelines on the basis of evidence or in situations where little or no evidence exists, based on the expectation that combining two or more interventions in an MCI is more effective than the single intervention (Reference Balinsky and Muennig2–Reference Prvu Bettger and Stineman4). Recommending an MCI on the basis of assumed incremental effectiveness over a single intervention is only reasonable when “doing more” indeed leads to better patient outcomes. However, it is not automatically the case that more interventions are better, nor that the combined effectiveness of more than one intervention follows an additive function. Reasons for this include interaction effects between interventions, which can cause an MCI that consists of two single interventions to have an effect that is greater or smaller than the sum of the two (Reference Smeets, Severens and Beelen5–Reference Lovemen, Frampton and Shepherd13).
As the financial pressure on health budgets compels consideration of cost-effectiveness data to inform reimbursement and implementation decisions, it is desirable to also take information on cost-effectiveness into account when recommending and using MCIs. For this purpose, the incremental cost-effectiveness of an MCI versus its single composite interventions ideally should be derived from clinical trials comparing the effects and costs of both the MCI and the single interventions against doing-nothing. However, as estimating the cost-effectiveness of all relevant interventions in clinical trials is expensive and time-consuming, the data on costs and effects of all interventions under comparison is in most cases incomplete and thus prohibits a full economic evaluation.
To still take cost-effectiveness considerations into account in the choice of an MCIs versus its single interventions, a pragmatic approach is needed that allows to estimate whether the MCI is more cost-effective than its single interventions in cases where empirical data on the effect and costs of the MCI are not (yet) or only partly available. This paper describes the development of such an approach and illustrates it using the example of the Dutch cancer rehabilitation guideline (1).
METHODS
The decision-problem is defined as follows: In the absence of sufficient data to evaluate an MCI's cost-effectiveness, how can it be estimated whether the MCI is likely to be more cost-effective than the single interventions it consists of? “Sufficient data” is data on key parameters required for analyzing the MCI´s (cost-) effectiveness with quality that meets the standards of evidence-based medicine and health economic evaluations (Reference Ramsey, Willke and Glick14–Reference Guyatt, Oxman and Vist16).
First, the literature was reviewed to identify methods for similar decision-problems that could contribute to developing an approach to solve the decision problem just defined. As it became clear during developing the approach that the most important part would be to estimate the effectiveness of the MCI, a large part of the search strategy was directed to this issue. Scopus, including literature from fields other than health care, was searched up to March 2017. Key words used were multidimensional, multicomponent, complex, multimodal, multifaceted, multi-treatment, program*, system, evaluation, assessment, prediction, impact, “estimat* effects”, influencing factors, treatment impact, sequential design, combined design, carry-over effect, interaction effects, cost-effectiveness, costs, economic evaluation, and effectiveness.
To qualify for inclusion, articles had to describe and/or apply a method that provided information for the development of the approach. Once included, the following information was retrieved descriptively from each of the articles: the methodological problem or the respective method addressed, the aim of the method, and a description of the method. If more than one paper on a respective method was identified, papers were included until saturation occurred. This was the case when one or more papers provided a comprehensive description of a method, to which further papers did not add new methodological aspects or were applications of the earlier presented method. If papers from leading professional associations were identified that contained methodology guidelines, no more articles for the respective method were included. This was done when additional articles did not provide more relevant details than the expert recommendations of associations in the field of health economics, such as the International Society for Pharmacoeconomics and Outcomes Research, or the Society for Medical Decision Making.
Moreover, we checked the comprehensiveness of the methodological recommendations before deciding not to expand further. After data extraction, the identified methods, the number of included articles reporting that method, and the type of articles per method found (methods description, methods development, application, or a mix of any of these), were summarized descriptively.
The requirements that the approach needs to fulfil were identified by analyzing the Dutch cancer rehabilitation guideline (1). For this purpose, the different pathways that patients may follow through the rehabilitation process based on the recommendations of the guideline were identified. Furthermore, we analyzed what an economic method needs to take into account to capture all aspects of the guideline-recommended care. These are:
-
(a) The guideline recommends an assessment of the patients’ needs and subsequently a tailored rehabilitation program, consisting of exercise, cognitive behavioral therapy (CBT), psycho-education, return-to-work, etc., or a combination of two or more of these interventions (1). As a large amount of intervention-combinations needs to be compared, the approach should be applicable within a limited time frame, considered to be within up to 5–15 hours for performing the approach, excluding time for information retrieval and assessment, identical to the typical timeframe for mini health-technology-assessments (mini-HTAs) (17). This also includes that the method can be applied by people who have knowledge in the field (of statistics, economics, health, etc.), but do not necessarily have completed postgraduate level training (e.g., PhD degree) or have conducted independent research previously.
-
(b) For many of the recommended combinations of interventions in this guideline, none or limited data regarding the costs and/or effects are available (1). Thus, it should be possible to carry out the approach when data on costs and effects of the MCI are limited.
-
(c) When single interventions are combined into an MCI, interaction and carry-over effects are likely to occur. In an interaction effect, the independent variable´s effect on the dependent variable depends on the level of another independent variable (Reference Senn18). Carry-over effects occur when participation in one intervention influences the performance of the participant in and/or the additional effectiveness of another intervention (Reference Senn18). Thus, the method needs to allow for taking into account that the effect of the MCI may depend on the combination of interventions, the order in which these are followed, and on the characteristics of the patients.
Subsequently, all methods that were identified in the literature review were evaluated on fulfilling requirement a. Those who passed were evaluated on fulfilling requirement b, and those who passed this step on requirement c. The decision of whether a method fulfilled the requirements was taken based on consensus among the authors. Method(s) fulfilling all requirements were analyzed on how to adapt them to fully suit the decision-problem. Finally, the developed approach is applied to an exemplar MCI from supportive care for cancer patients. It was illustrated how the approach can be used to estimate what is expected to be more cost-effective, the MCI or one of the composite interventions.
RESULTS
Literature Review
In the review, thirty-six papers were included, from which eight methods were extracted. These methods are: (i) assessing the influence of mediating factors, (ii) statistical modeling of the (cost-)effectiveness of interventions (such as extrapolation and transfer of model results to other settings, logistic regression analysis, whole disease modeling, etc.), (iii) retrospectively identifying success factors of interventions, (iv) using decision-rules for estimating the effectiveness of multicomponent interventions based on its single components, (v) network meta-analyses/indirect treatment comparisons, (vi) headroom analysis/cost-effectiveness gap analysis, (vii) process evaluation, and (viii) realist randomized controlled trials (RCTs). The methods are described in Table 1.
Methods Identified in the Literature Review and Their Descriptions Identified in the Literature Review
MCI, multicomponent intervention; ICER, incremental cost-effectiveness ratio; RCT, randomized controlled trial; WTP, willingness-to-pay; MRIE, minimum required incremental effect.
Development of the Stepwise Approach
Choice of a Method
Table 2 shows which methods fulfilled the requirements. Requirement a, being able to estimate the cost-effectiveness of an MCI versus its single interventions, was fulfilled by two methods: headroom analysis and using decision-rules. The remaining methods require statistical data analysis, which is time-consuming, especially when conducted for several MCIs, and require advanced statistical expertise. Requirement b, being able to estimate cost-effectiveness in the absence of full data on incremental effects and costs, was fulfilled by both the methods that also fulfilled the first requirement. Requirement c, being able to take carry-over and interaction effects into account, was fulfilled by both methods as well.
Analysis of which Methods Fulfil the Requirements
MCI, multicomponent intervention; RCT, randomized controlled trial.
For decision-rules, the degree to which this criterion is fulfilled depends on the specific decision rule that is used, as a decision rule could, for example, be to assume an effect larger than the sum of the interventions’ effects when synergy effects are expected. Of these two methods, headroom analysis was chosen and used as a starting point for further development into an approach as it enables comparison of the cost-effectiveness of the MCI with that of its composite interventions; in addition, the willingness-to-pay (WTP) is included in this method because it enables us to make clear decisions about whether an intervention is cost-effective or not.
The basis of headroom-analysis is the net-benefit framework, which is used to determine the incremental cost-effectiveness ratio (ICER) of an intervention.
$$\begin{equation}
ICER = \left( {\Delta Cost \div \Delta {\it Effect}} \right).
\end{equation}$$
The incremental costs of an intervention are divided by its incremental effect to determine what the additional cost for obtaining one additional unit of effect are.
Headroom-analysis is used in cases in which either the incremental costs or the incremental effect of an intervention are unknown. The pivotal parameter in the headroom method is the maximum amount someone is willing to spend on one unit of additional effect, that is, WTP. Then, depending on which of the other two parameters is known, that is, either the minimum required incremental effect (MRIE) an intervention needs to attain, or the maximum costs it may have to be considered cost-effective can be calculated with the following equations:
$$\begin{equation}
MR\Delta E \ge \left( {\Delta Costs \div WTP} \right)
\end{equation}$$
$$\begin{equation}
Max\Delta C \le \left( {\Delta {\it Effect} \times WTP} \right)
\end{equation}$$
where MRΔE = MRIE; MaxΔC = maximum incremental costs.
The next step is to estimate to what extent the respective intervention is likely to attain the required effectiveness or to stay below a maximum cost. For being fully suitable to solve our decision-problem, headroom analysis should explicitly accommodate the estimation of whether the MCI can attain the minimum effect or stay below the maximum costs, and compare this with the cost-effectiveness of the single intervention. The “conventional” headroom analysis does not explicitly allow for this, so this was developed as part of this study and described below.
Development of the Approach
As costs of an intervention can be calculated relatively easily, that is, in comparison to the effectiveness, this approach focuses on whether the MRIE would be attained. To accommodate MCIs in the headroom analysis-approach, the incremental effect and costs of a single intervention are replaced by those of the MCI in Eq. [2]. The MCI is labeled “AB” and consists of the single interventions “A” and “B”.
$$\begin{equation}
MR\Delta {\it Effect}\ of\ AB \ge \left( {\Delta CostAB \div WTP} \right).
\end{equation}$$
Afterward, a qualitative assessment of how likely the respective intervention is to attain the MRIE, given the incremental effects or costs and the WTP, needs to be performed. Then, it needs to be estimated if the MCI is likely to attain the MRIE, and how this compares with the cost-effectiveness of the single interventions. It is not a precondition for the approach that the single interventions are cost-effective by themselves, as the MCI can be cost-effective when super-additive effects occur.
The Stepwise Approach
The approach consists of the following six steps:
1. Estimate Intervention Costs: When the costs of the MCI are not empirically investigated, these can be estimated using the cost calculation of the single interventions. It needs to be analyzed which costs are (super-) additive when the composite interventions are merged into an MCI, and which costs are redundant or sub-additive (Reference synergy19;Reference Hollingsworth and Peacock20). Furthermore, detailed intervention descriptions can be used and, when required, the intervention developers may be willing to share more details and data upon request.
2. Choose a Common Outcome Measure: Because the incremental effectiveness of the interventions under comparison is expressed in one outcome measure, of which the monetary value is determined through the WTP, the effectiveness of all interventions needs to be expressed in that same outcome measure. Most suitable for economic evaluations are generic outcome measures, such as quality-adjusted life-years (QALYs).
3. Determine the WTP: The WTP for the selected outcome measure, that is, the maximum amount society is willing to pay for an additional unit of effect, needs to be determined.
4. Analyze the MRIE: The MRIE of the MCI needs to be assessed by completing Eq. [4], using the WTP and the cost of the MCI as determined in step 1.
5. Qualitatively Assess How Likely the MCI Is to Attain the MRIE: The potential for the MCI to attain the required MRIE needs to be assessed. For this, the following information and any other information that is considered relevant to assess the potential effect of the MCI should be gathered from experts or identified in the literature (as far as available): (i) The effectiveness of the single interventions, as this indicates the potential effect size. (ii) The interaction and carry-over effects that are assumed to occur in an MCI, and whether these diminish, enlarge, or do not influence the MCI's effectiveness. (iii) The intensity of the intervention (e.g., in physical exercise [PE] interventions this could be walking versus running). More intense interventions may lead to larger effect sizes. Too high intensity might lead to exhaustion and diminish the effect of the MCI. (iv) The effect-causing mechanisms of the single interventions and whether these differ. When this is the case, it can be assumed that the effect is larger than that of one intervention. (v) The effect of similar (combinations of) interventions or of the respective intervention in another patient group can give an impression of the reachable effect. (vi) The symptom burden at baseline. A higher burden may lead to a greater potential for achieving a high effect. (vii) The natural course of the symptoms. If, for example, the symptoms fade out by themselves, the potential for a high effect of the MCI is smaller than when symptoms would increase without any treatment. (viii) The duration of the treatment effect. As QALYs are a function of utility and the duration in which the utility is higher than in the comparison group, this has a high influence on the QALY-gain.
Based on these aspects, a qualitative assessment could be conducted of how likely the MCI is to attain the MRIE; for example on a 4-point Likert-scale defined as “very likely,” “likely,” “unlikely,” or “very unlikely.”
6. Compare the cost-effectiveness of the single interventions with the potential of the MCI for being cost-effective: The potential for the MCI to be cost-effective depends on the cost-effectiveness of the single interventions. If the MCI is likely or very likely to be cost-effective, but the single interventions are not, the MCI should be adopted. In turn, when one or both single interventions are cost-effective, but the MCI is unlikely or very unlikely to be cost-effective, the most cost-effective single intervention is recommended. In Table 3, it is shown in which cases the MCI or the single interventions should be adopted.
Which Intervention Is Identified as Most Cost-Effective?
a In case of “undecided,” the choice can be based on patient preferences, on the effectiveness of the interventions, or on the budget impact. Another possibility is to repeat the approach with a willingness-to-pay that resembles the ICER of the (most) cost-effective single intervention. In that way, it can be analyzed if the MCI still is considered cost-effective when the willingness-to-pay is as high as the ICER of the single intervention.
MCI, Multicomponent intervention; ICER, Incremental cost-effectiveness ratio.
Application of the Stepwise Approach
An example from supportive cancer care is used as an illustration, which consists of interventions for alleviating treatment-induced menopausal symptoms in breast cancer patients. The MCI consists of two single interventions, CBT and PE, which are both also prescribed as stand-alone interventions (Reference Duijts and Van Beurden M21). CBT has a cost of €190 and adds 0.0079 QALYs compared with usual care, PE costs €197 and adds 0.0067 QALYs (Reference Mewes, Steuten and Oldenburg22). The ICER for CBT is €24,050/QALYS and €29,402/QALY for PE.
1. Estimate intervention costs. For the MCI, the costs of CBT and PE are additive, that is, the sum of those for PE and for CBT; €387. They are not sub-additive, as the time for planning appointments and the required resources for providing the interventions remain the same.
2. Choose a common outcome measure. QALYs are chosen as the common outcome measure for all interventions involved in the comparison, as these are recommended for health economic analyses.
3. Determine the WTP. WTP is assumed at €30,000 per QALY.
4. Analyze the MRIE. The MRIE for the MCI consisting of both PE and CBT is:
$$\begin{eqnarray*}
&& Min\Delta {\it Effect}A + Min\Delta {\it Effect}B \ge \left( {\left( {\Delta CostsA + \Delta CostsB} \right) \div WTP} \right) \\
&& Min\Delta {\it Effect}A + Min\Delta {\it Effect}B \ge \left( {\euro 387 \div 30,000} \right) = 0.0129
\end{eqnarray*}$$
5. Assess if the MCI and the composite interventions are likely to attain the MRIE. For assessing if the interventions are likely to attain the MRIEs, the following information was considered: (i) To attain the MRIE, an effect of almost the sum of CBT and PE is required. (ii) When combining CBT and PE, it is likely that an interaction effect occurs. For participating in the MCI, patients need to travel to the hospital at least twice a week, which might be exhausting and could reduce the effect of the interventions as patients might be more tired. (iii) CBT and PE have different effect-causing mechanisms (Reference Duijts and Van Beurden M21;Reference Duijts, Oldenburg and van Beurden23–Reference Hunter26). While CBT aims to influence the perception of the symptoms, PE is assumed to actually reduce the severity and frequency of symptoms (Reference Ivarsson, Spetz and Hammar24–Reference Hunter26). Therefore, the MCI is expected to be more effective than CBT or PE alone. (iv) The baseline quality-of-life was 0.78 (Reference Mewes, Steuten and Oldenburg22), indicating a relatively low symptom burden. (v) Menopausal symptoms fade out during a period of approximately 5 years (Reference McKinlay, Brambilla and Posner27), leading to an average maximum effect duration of 5 years. (vi) The evidence on the effectiveness of exercise for reducing vasomotor symptoms is inconclusive (Reference Duijts, Faber and Oldenburg28). A meta-analysis found an effect size of 0.3 of exercise on health-related quality of life in breast cancer patients (Reference Duijts, Faber and Oldenburg28). (vii) The effect of the MCI cannot last longer than the 5 years for which the symptoms normally are experienced. Moreover, the potential effect size becomes smaller throughout the years, as the symptoms diminish naturally.
It was estimated that the MCI is unlikely to reach the MRIE of 0.0129. The MRIE is almost the sum of the effects of PE and CBT, as no sub-additive effects for the costs occur. The main reason for the conclusion is that the symptom burden of the patients was relatively low. As the symptoms fade out by themselves and the single interventions PE and CBT do not result in effect sizes of that magnitude, the potential for reaching the high MRIE is small. CBT+PE is assumed to have a larger effect than one of them alone, as they have different effect-causing mechanisms. However, the intensity of following both interventions is assumed to be tiresome for some participants, which would further diminish the combined effect, so that it is unlikely to reach the sum of PE and CBT.
6. Compare the potential for cost-effectiveness of the MCI with that of the composite interventions. The single interventions CBT and PE both are cost-effective. The MCI may add further effectiveness; however, it is estimated that the MRE of 0.0129 QALYs is unlikely to be attained. Thus, when aiming to choose a cost-effective intervention, one of the single interventions should be preferred.
DISCUSSION
We developed and illustrated an approach that enables to estimate which intervention is likely to be more cost-effective, an MCI or its composite interventions, when empirical data on costs and effects of the MCI are incomplete. It is meant to be pragmatic, and feasible to perform within a short time frame and without advanced health economic expertise, as this would limit its use in practice by decision makers. The approach builds on headroom analysis, which was extended to accommodate MCIs. Given the costs of the MCI and the WTP, the MRIE of the interventions to attain cost-effectiveness can be assessed. Subsequently, a qualitative yet systematic assessment of how likely the MCI is to achieve the MRIE can be made. Depending on whether the single interventions are cost-effective, it can be assessed whether the MCI is likely to be more cost-effective.
The literature review showed that methodology concerning this specific decision-problem is rather scarce. Reasons for this gap are that guideline developers traditionally have been more concerned with effectiveness than cost-effectiveness, that including economic evidence into guidelines makes the process of developing them even lengthier and more resource intensive (Reference Eccles and Mason29), and that clear criteria of when an intervention is considered cost-effective are lacking. One approach to overcome this is the framework by Gandjour and Lauterbach (Reference Gandjour and Lauterbach30) for assessing the break-even point of guidelines. Our approach adds the option to estimate the potential for cost-effectiveness of an MCI in the absence of full data and in a pragmatic way that requirement c be conducted by decision makers.
The approach presented here is developed for national decision making, but with little adaptation it can also support local decision makers. Hospitals or healthcare providers can use it to decide on how to implement a comprehensive guideline advice by replacing the WTP by their budget or the reimbursement to analyze how to achieve the best use of the available resources. This would form a useful addition to the current mini-HTAs, which are used, for example, for making decisions on local budgets (17).
In many countries and in many fields in health care, such as public health, a fixed cost-effectiveness threshold or WTP does not exist and a range of threshold values is considered. In that case, the approach can be conducted several times using a variety of thresholds (e.g., €20,000 to €100,000 per QALY). By doing so, the range of thresholds in which the intervention or technology is expected to be cost-effective can be identified. This can then be used as one among other criteria in the decision to fund an intervention.
Another variant of the approach is to not analyze the MRIE and then decide if the intervention potentially can attain this, but to first estimate the range of the effect that the intervention is thought to attain, and to then determine the range of ICERs that this corresponds to. Subsequently, the intervention with the most favorable ICER range can be chosen.
The approach was developed for healthcare interventions and primarily based on literature from this field. However, it also has relevance to fields outside of healthcare, such as the evaluation of the societal costs of (various health and nonhealth) interventions that impact the environment. Moreover, literature and experiences from this and other fields should also be used when further developing the methodology for evaluating the cost-effectiveness of multicomponent interventions (31–Reference Heinzerling and Ackerman33).
The key missing information in the puzzle of the cost-effectiveness is the possible interaction effect of the MCI. The more (separate) information is available that informs the estimation of the size of the effect, the more precise the estimation of the combined effect is. To improve the accurate estimation of the effect size, future research could be focused on discriminative analysis of the effect of components of care versus that of the combined approach. One direction that could be taken is to use the various approaches that sensitivity analysis offers, which also take the already existing information and clinical expertise into account (Reference Liu, Kuramoto and Stuart34). Another direction is the approach by Basu et al. in which prediction models are used that analyze the utility of a joint health state, that is, a patient experiencing several symptoms, from the utility of single health states, that is, from patients having only one symptom (Reference Basu, Dale and Elstein35).
Moreover, to evaluate the benefit of the develop approach in practice, future research is needed. Comparisons should be made of the results of this approach versus those of an economic evaluation alongside an RCT to gain more insight into the validity of the results of headroom analysis for multicomponent interventions.
CONCLUSION
We developed a systematic approach to estimate, in the absence of suitable data on costs and effects, if an MCI can be more cost-effective than its composite, single interventions. The purpose of this method is to support the decision of which interventions to recommend in guidelines based on cost-effectiveness considerations, in the absence of full (and timely) data. We illustrated the use of this method in the field of cancer survivorship. Further research is needed in which headroom analysis for MCIs is applied to multicomponent guidelines, to test the method´s practical use and impact.
CONFLICTS OF INTEREST
The authors confirm that they do not have a conflict of interest.
