Current practice: using the Baron and Kenny method

Baron and Kenny’s⁽Reference Baron and Kenny³⁾ seminal paper on mediators and moderators is by far the most cited published piece of work within studies investigating mediators of nutrition and physical activity behaviour. According to their not always correctly interpreted conceptual framework, to function as a mediator, a variable must meet four criteria.

1. 1. An independent variable (e.g. fat-intake intervention) must be significantly associated with the potential mediator (e.g. change in self-efficacy for a low-fat diet); this relationship is operationalized as the α regression coefficient (also termed ‘path coefficient’ in the language of structural equation modelling; see Fig. 1).

   Fig. 1

   Diagram for analysis of the effect of a fat-intake intervention on changes in fat intake mediated by self-efficacy for a low-fat diet

   

2. 2. The potential mediator (self-efficacy) must be significantly associated with the outcome (e.g. change in fat intake), after adjustment for the independent variable; this is represented by the β coefficient (see Fig. 1).

3. 3. After adjustment for the potential mediator (change in self-efficacy), a previously significant relationship between the outcome (change in fat intake) and the independent variable (fat-intake intervention) is no longer significant or is significantly attenuated; this is represented by the τ′ coefficient (see Fig. 1).

4. 4. The third condition implies a significant relationship between the outcome (change in fat intake) and the independent variable (fat-intake intervention), which constitutes the fourth criterion; this relationship is represented by the τ coefficient (see Fig. 1). All of these coefficients are best obtained using regression models or structural equation modelling.

In the domain of behavioural prevention programmes, an evaluation of the first relationship (α coefficient) is termed ‘action theory test’ and informs us on the extent to which we were successful in manipulating the hypothesized mechanisms of behaviour change. The second relationship (β coefficient) represents a ‘conceptual theory test’, i.e. the extent to which there is empirical support for our conceptual theory (e.g. Social Cognitive Theory) that changes in the hypothesized mechanisms cause changes in behaviour⁽Reference Chen⁴⁾. The italicized text above highlights important aspects of MVA that are sometimes overlooked by researchers claiming to follow Baron and Kenny’s method.

Misinterpreting Baron and Kenny

It is not uncommon to see studies published in very respectable journals reporting a significant association between the mediator and the outcome, unadjusted for the effects of the independent variable, as one of the criteria for mediation⁽Reference Miller, Trost and Brown^5–Reference Prodaniuk, Plotnikoff, Spence and Wilson⁸⁾. The source of confusion appears to be Baron and Kenny’s unfortunate omission of the words ‘controlling or adjusting for the independent variable’ in the first part of their well-cited article (p. 1176), while later (p. 1177) they clearly explain that the second criterion of mediation (see above) needs to be tested by ‘regressing the dependent variable on both the independent variable and on the mediator’⁽Reference Baron and Kenny³⁾. The reason for this necessary adjustment is that the mediator and the outcome may be related because they are both caused by the independent variable (in which case, there is no mediation) and not because the independent variable influences the mediator, which then influences the outcome⁽Reference Kenny, Kashy and Bolger⁹⁾. Consequently, our first take-home message is that to test mediation according to the Baron and Kenny method it is unnecessary to examine the unadjusted relationship between the mediator and the outcome. However, it is necessary to establish that the mediator is related to the outcome independently of the independent variable.

Beyond the four criteria of mediation: in search for a measure of mediated effect

Although a formal statistical test of significance of the mediated effect (sometimes called ‘indirect effect’) is not one of the explicit criteria of mediation proposed by Baron and Kenny, they nevertheless note that a significantly attenuated independent variable–outcome relationship, following adjustment for the potential mediator, is a necessary condition for mediation. The magnitude of the attenuation is a measure of the mediated effect. The question is: what do we mean by significant mediated effect (or attenuation) and why is this information important? We argue that truly ‘significant’ mediators are those that satisfy both requirements of statistical and substantive or clinical significance. While statistical significance ensures that we identify reliable mediators, substantive or clinical significance ensures that we focus our efforts on finding and manipulating mechanisms that produce the largest effects in terms of behaviour or health outcomes, thus maximizing the cost-effectiveness of our interventions. Here is where the vast majority of published work runs into problems.

Let us start with statistical significance. There are many ways in which we can test the statistical significance of a mediated effect⁽Reference MacKinnon^10, Reference MacKinnon, Fairchild and Fritz¹¹⁾. In general, if a variable meets all four criteria of mediation as per Baron and Kenny, we can confidently conclude that it is a reliable mediator. In fact, simulation studies have shown Baron and Kenny’s method to be one the most conservative tests of mediation⁽Reference MacKinnon, Fairchild and Fritz^11, Reference MacKinnon, Lockwood, Hoffman, West and Sheet¹²⁾. However, as explained later in more detail, the downside of this overly conservative approach is that, in some circumstances, it can fail to identify reliable and substantively meaningful mediators⁽Reference Cerin, Taylor, Leslie and Owen¹³⁾. On the other hand, large-scale studies may produce statistically significant but substantively or clinically trivial mediating effects. This is why it is important for studies adopting this particular MVA framework to go beyond the four criteria of mediation and explicitly report the observed mediated effect for each examined mediator. This needs to be done using simple, single-mediator as well as complex, multiple-mediator models evaluating the independent contribution of each mediating variable. Importantly, such practice is essential if solid knowledge about mechanism and strategies of behaviour change is to be swiftly accumulated and synthesized with the aid of meta-analytical procedures. Any alternative course of action is bound to unnecessarily slow down the progress in this area of research.

A measure of a mediated effect can be obtained by computing the product of α coefficient (independent-variable effect on the mediator) and β coefficient (mediator effect on the outcome adjusted for the independent variable; see Fig. 1). This is termed ‘product-of-coefficient estimate’ of a mediated effect⁽Reference MacKinnon^10–Reference MacKinnon, Lockwood, Hoffman, West and Sheet¹²⁾. When using ordinary least squares regression in single-mediator models, it is algebraically equivalent to computing the reduction in the independent-variable effect after inclusion of the mediator in the regression model⁽Reference MacKinnon^10–Reference MacKinnon, Lockwood, Hoffman, West and Sheet¹²⁾. This is named a ‘difference-of-coefficients estimate’. These estimates of mediated effect represent measures of the effect of the independent variable on the outcome via the mediator (the proposed mechanism) in the units used to measure the outcome (which implies that unstandardized regression coefficients have been used to compute them); hence their appeal and informative value.

For instance, if an intervention produced a decrease in fat intake of 10 g/d, and after including self-efficacy for a low-fat diet in the model this effect is reduced to 4 g/d, we would state that the effect of the intervention mediated by self-efficacy was a decrease in fat intake of 6 g/d. In other words, the intervention yielded a reduction in fat intake of 6 g/d likely because of its effect on self-efficacy. In terms of product-of-coefficient estimate, if the intervention has an average effect on self-efficacy of 2 units on a 5-point scale (α coefficient) and the independent effect of self-efficacy is a decrease in fat of 3 g/d per unit increase in self-efficacy (β coefficient; conceptual theory test), by multiplying these estimates we obtain a mediated effect of the intervention through self-efficacy of 2 × (−3) g/d=−6 g/d (i.e. a decrease in fat intake of 6 g/d). These values inform us of the extent to which self-efficacy is important in changing the target behaviour (3 g/d per unit change in self-efficacy; conceptual theory test), our ability to change self-efficacy (a 2 unit change on a 5 point-scale; action theory test), and the expected benefit of targeting self-efficacy with a specific type of intervention within a specific target population (a decrease in fat intake of 6 g/d). Hence, the second take-home message of the present paper, especially relevant to intervention studies, is that it is important to report estimates of mediated effects as well as action and conceptual theory tests (see Haerens et al.⁽Reference Haerens, Cerin, Maes, Cardon, Deforche and De Bourdeaudhuij¹⁴⁾ for an example).

Beyond the four criteria of mediation: a special note on the ‘dichotomous’ case

Dichotomous outcome variables, such as being overweight v. being of normal weight or meeting v. not meeting the physical activity recommendations for general health, are often used in behavioural nutrition and physical activity research. In such cases, assuming that the mediator is a continuous variable, a mediating effect can be computed using a product-of-coefficient test based on the estimate of α from ordinary least squares regression and β from logistic or probit regression⁽Reference MacKinnon^10, Reference MacKinnon, Fairchild and Fritz¹¹⁾. This method has been found to be robust against departures from the logistic or probit assumptions as well as normality assumptions for the distribution of the mediator⁽Reference MacKinnon, Lockwood, Brown, Wang and Hoffman¹⁵⁾. Importantly, for dichotomous outcomes, the difference-of-coefficient method yields incorrect results and, hence, its use is not recommended⁽Reference MacKinnon, Fairchild and Fritz^11, Reference MacKinnon, Lockwood, Brown, Wang and Hoffman¹⁵⁾. The same applies to the commonly used approach to MVA based on an examination of the presence of a reduction in the regression coefficient of the independent variable (e.g. intervention) after inclusion of the mediator in the logistic regression model (criterion 3 of the Baron–Kenny method). Specifically, when using these methods, larger actual mediated effects may paradoxically result in smaller or nil estimates of the mediated effect or a reduction in coefficients. This effect is due to the difference in scales of the logistic regression coefficients of the outcome on the independent variable adjusting and not adjusting for the mediator, since these coefficients are derived from separate logistic regression equations (with a fixed error variance). Hence, the potential negative consequences of using Baron and Kenny’s method to conduct MVA with a dichotomous outcome are particularly serious. The third take-home message of the paper is to use the product-of-coefficient method when dealing with dichotomous outcome variables.

Beyond the four criteria of mediation: confidence intervals

Any statistical estimate of effect is accompanied by a measure of its accuracy (i.e. a standard error), which allows for the computation of the range of plausible values (i.e. the confidence interval) of the population parameter (i.e. mediated effect) supported by the data. Confidence intervals tell us how confident we can be that the observed estimate of effect corresponds to the effect in the target population. They provide clearer information than the usual P values because the latter are a confounded mixture of effect size and sample size, while the width of the former is determined by sample size and variability in effect of interest (determining accuracy). This also applies to the estimate of the mediated effect. Using the fat-intake example, a 95 % confidence interval of fat intake ranging from −1 to −11 g/d would indicate that the plausible population effects of the intervention on fat intake through self-efficacy are a decrease of fat intake from 1 to 11 g/d.

The information given by the point estimate and confidence limits of the mediated effect is very useful in guiding efforts to enhance theory and practice of behaviour change. For instance, a clinically meaningful mediated effect accompanied by large confidence limits indicates four main possible sources of problems: (i) insufficient sample size; (ii) large measurement error; (iii) collinearity between the independent variable and the mediator affecting the accuracy of the β coefficient estimate⁽Reference Baron and Kenny³⁾; (iv) large inter-individual variations in the effects of the intervention on the mediator or/and the effects of the mediator on the outcome. In contrast, a clinically trivial but accurate (with a narrow confidence interval) estimate of mediated effect points directly at problems with action (the effectiveness of our intervention in changing the mediator) and conceptual theory (theoretical mechanisms of change). Thus our fourth take-home message is: report confidence intervals of mediated effects and interpret them conjointly with their point estimates.

How do we obtain confidence limits of mediated effects? Baron and Kenny⁽Reference Baron and Kenny³⁾ provided a formula for the standard error of the mediated effect based on Sobel⁽Reference Sobel¹⁶⁾. The use of the formula assumes that the mediated effect is approximately normally distributed. This solution is less than optimal because, unless the ratio of the α and/or β coefficients to their standard errors is equal or greater than 6, mediated effects are not normally distributed⁽Reference MacKinnon, Lockwood, Hoffman, West and Sheet^12, Reference Bollen and Stine^17–Reference Shrout and Bolger¹⁹⁾. However, it is very easy to implement by those less versed in more sophisticated methods of MVA. All it requires are the point estimates and standard errors of α and β coefficients (see Preacher’s (2003) online program at http://www.psych.ku.edu/preacher/sobel/sobel.htm and associated ‘warnings’).

More optimal and recommended solutions involve the computation of asymmetric confidence intervals of mediated effects using a recent program developed by MacKinnon et al.⁽Reference MacKinnon, Fritz, Williams and Lockwood²⁰⁾ (which can be found at http://www.public.asu.edu/~davidpm/ripl/Prodclin) and the use of bootstrapping resampling techniques⁽Reference Bollen and Stine^17, Reference Shrout and Bolger^19, Reference Preacher and Hayes²¹⁾. Bootstrapping allows the distribution of the mediated effect to be estimated empirically by using information from the original sample (treated as a pseudo population)⁽Reference Efron and Tibshirani²²⁾. It is particularly recommended for non-normally distributed variables, such as mediated effects, for which parametric inferential statistics may produce biased estimates. This method consists in drawing with replacement a large number of bootstrap samples (e.g. 1000) from the original sample; estimating mediated effects for each of these samples; averaging the effect estimates across all of the bootstrap samples; and computing the 95 % percentile confidence intervals for the mediated effects across all of the bootstrap samples⁽Reference Efron and Tibshirani²²⁾. Mediators whose confidence limits do not include zero are considered reliable (statistically significant). It is also recommended that the more accurate bias-corrected bootstrap confidence intervals, corrected for the difference between the original estimate and the bootstrap mean estimate, be computed⁽Reference Pituch, Stapleton and Kang²³⁾ (see Cerin and Leslie⁽Reference Cerin and Leslie²⁴⁾ for examples in the field of physical activity).

Is it really necessary to test the association between the independent variable and the outcome in mediating variable analysis?

The ultimate aim of MVA is to establish whether an independent variable influences the outcome through its effect on a mediating variable. If this is the case, do we really need to show that the independent variable is related to the outcome (fourth criterion of mediation as per Baron and Kenny)? The answer is no. We might be legitimately interested in the overall association of, for instance, socio-economic status and physical activity behaviour. However, a significant association between the two is not a requirement for mediation to occur. Actually, this particular criterion can hinder the discovery of substantive mechanisms of influence.

There are three major reasons for this. The first reason, discussed by Judd and Kenny⁽Reference Judd and Kenny²⁵⁾ and more recently explained by Kraemer et al.⁽Reference Kraemer, Wilson, Fairburn and Agras²⁶⁾, relates to the fact that the effect of the mediator on the outcome may depend on the values of the independent variable (interaction effect). It is theoretically possible for an intervention to leave unaltered the average level of the outcome (τ coefficient) and mediator (β coefficient) but change the direction of the effect of the mediator on the outcome (see Fig. 2a; β _C and β _T coefficients). In such a case, mediation would occur without observing a significant intervention effect on the outcome. As a matter of fact, there might not even be any significant intervention effects on the mediator or overall independent effect of the mediator on the outcome (Fig. 2a).

Fig. 2

Examples of mediation in the absence of a significant association between the independent variable and the outcome: (a) interaction between an intervention and the mediator; (b) inconsistent mediation

The second reason regards the possibility that the effect of an independent variable on the outcome be explained by multiple competing mechanisms whose influences are of opposite direction and cancel out⁽Reference Shrout and Bolger^19, Reference MacKinnon, Krull and Lockwood²⁷⁾. For instance, it has been shown that an intervention-induced increase in physical activity (mediator of intervention effect, if the aim is to reduce BMI by increasing energy expenditure) in normal-weight adults does not generally result in a decrease in BMI (outcome) because it is also accompanied by an increase in energy intake (counterproductive or inconsistent mediator of intervention effect)⁽Reference Meltzer, Kayser, Saris and Pichard²⁸⁾. The intervention triggers two different mechanisms exerting opposite effects on BMI. It is clear that here mediation occurs even in lack of a significant overall effect of the intervention on the outcome (see Fig. 2b).

Inconsistent mediation, also called suppression, can sometimes be counterintuitive (see MacKinnon et al.⁽Reference MacKinnon, Krull and Lockwood²⁷⁾ for a discussion on inconsistent mediators). In such cases, assuming a robust effect, it is a warning that there may be serious problems with: (i) our action and conceptual theories; or (ii) our mediating variables measures. For example, unexpected inconsistent mediation effects have been recently reported in physical activity and nutrition intervention studies⁽Reference Haerens, Cerin, Maes, Cardon, Deforche and De Bourdeaudhuij^14, Reference Haerens, Cerin, Deforche, Maes and De Bourdeaudhuij²⁹⁾. Self-efficacy and perceived barriers were found to be independently associated, in the theoretically predicted direction, with changes in physical activity and fat intake, respectively. However, the intervention was found to have a negative effect on self-efficacy and perceived barriers, resulting in inconsistent mediation effects. In other words, the intervention would have been more successful had not it exerted a negative effect on self-efficacy and perceived barriers as mechanisms of behaviour change. Changes in self-efficacy and perceived barriers suppressed the intervention effectiveness. The authors concluded that their strategies to manipulate mediators of behaviour change in adolescents needed to be reconsidered (problems with action theory) and/or that an intervention by measurement effect might have occurred, whereby exposure to the intervention affected the interpretation of the items in relation to the underlying construct (change of location of the items on the underlying construct). These explanations can be verified by employing differential item functioning (DIF) techniques in combination with having objective behavioural markers of self-efficacy and perceived barriers to contrast with the DIF of the scale items⁽Reference Baranowski, Allen, Masse and Wilson³⁰⁾.

The third reason for not considering a significant relationship between an independent variable and the outcome as a necessary criterion of mediation is low statistical power⁽Reference MacKinnon, Lockwood, Hoffman, West and Sheet^12, Reference Fritz and MacKinnon^18, Reference Shrout and Bolger¹⁹⁾. This argument is particularly relevant if we believe that the effect of the independent variable on the outcome is temporally or causally distal⁽Reference Shrout and Bolger¹⁹⁾. In their simulation studies, Fritz and MacKinnon⁽Reference Fritz and MacKinnon¹⁸⁾ have shown that, under conditions of complete mediation and small effect size, we would need a sample size of approximately 21 000 cases to achieve acceptable levels of statistical power. In contrast, the sample size required to detect the same mediated effect using product-of-coefficient tests (e.g. Sobel test, bootstrap estimates and asymmetric confidence intervals) would range between 490 and 660 cases.