As discussed previously Reference Hadzi-Pavlovic(1), in a clinical trial examining the difference between a new treatment (T) and a control treatment (C), we often compare subjects on measurements made at two different times. Typically, PRE scores (T 1 and C 1) are measured before treatment; and POST scores (T 2 and C 2) are measured at the end of treatment. Of course, PRE and POST (in particular) are flexible and can represent any two time-points: POST could be equally a 3-month follow-up, or some point during treatment, which is being compared with pre-treatment.
One way of expressing the difference between treatments is to ask if the patients differ on their POST scores, that is, does the mean of T 2 differ from the mean of C2. Another way is to ask if the patients differ on their CHANGE (PRE minus POST) scores, that is, does the mean of T Δ = T 1–T 2 differ from the mean of C Δ = C 1–C 2.
In a clinical trial with random allocation of treatment, PRE scores are likely to be quite similar, and the size of any difference in POST or CHANGE scores can usually be attributed to the relative efficacy of the different treatments. Sometimes, however, PRE scores are not similar and researchers ask whether this might explain the final differences. If patients receiving T are more severe than those receiving C, for example, then this might explain why the new treatment appears not better than the control. The analysis of covariance (ANCOVA) is frequently used to answer this question.
As discussed in Reference Hadzi-Pavlovic(2), the effect of ANCOVA is to shift the means further apart or closer together based on the regression of the outcome on the covariate(s) and therefore change the result of a t-test comparing the means.
In Table 1, we outline the four analyses that a researcher chooses from by deciding whether to analyse POST or CHANGE scores and whether to include PRE as a covariate or not.
Options for analysis of PRE–POST data
Although there are four options given above, it has been known for some time that analyses II and IV give exactly the same result, that is, adjusting POST for PRE and adjusting CHANGE for PRE give the same result for the difference between the two treatments, even though the regression of POST on PRE differs from that of CHANGE on PRE. This equivalence continues if we add other covariates to PRE, so that if we additionally adjust, for example, for age and sex, the difference between treatments on the two outcomes is again the same. Again the regressions of the covariates (including significance) will not be the same: that of POST on age will not be the same as that of CHANGE, just as they differ regressed on PRE.
Figures 1 and 2 are based on three situations: where C is better than T at baseline, where they are the same and where C is worse. At POST, the means are C = 24 and T = 18. For each of these situations, we look at how the results of an ANCOVA alter as the PRE–POST correlation goes from–0.7 to +0.7.
Effect of PRE–POST correlation on adjusted POST means under various conditions (see text for details).
Effect of PRE–POST correlation on p-values for t-test of adjusted POST means under various conditions (see text for details).
Figure 1 shows how the adjusted means change accordingly as our conditions change. When the PRE–POST correlation is zero, there is of course no change (the adjusted means equal the observed means). As the correlation increases positively, the means move further apart if T was higher than C at baseline and they come closer together if C was higher than T. One way of thinking about this is to note that if T was higher than C at PRE and is lower at POST, then there must have been an even greater change under T than under C. Similarly for the other conditions, but reversed where applicable.
Figure 2 shows how the p-values change from the t-test (they are expressed as percentages for graphing purposes). When there is a difference in PRE (left and right panels), we see how the t-test becomes increasingly significant (non-significant) as the correlation moves from–0.7 to +0.7. The benefits of the PRE–POST correlation are shown even though the observed means do not change.
Even when there is no difference at PRE (centre panel), and so no difference in the adjusted means, the p-value still changes symmetrically. The reason for this is that precision of our estimates is increased by the increasing correlations, and this was one of the original uses of ANCOVA in experimental research where initial differences are usually negligible.
Figures 3 and 4 are based on the situation where the PRE means are C = 32 and T = 36 and the PRE–POST correlation is 0.5. We then look at what happens as the POST mean for C remains constant at 24 while the POST mean for T changes from 20 (T is better) to 28 (T is poorer). In Fig. 3, we plot the adjusted POST scores. Note how because T was worse at baseline (by 4 points) it only becomes worse by the adjusted scores when T−C is 3 or more at post-treatment and not when it is 0 or more. In Fig. 4, we plot the p-values from the t-test by comparing the adjusted means. T remains significantly different from C until just before the difference reaches zero. Thereafter, the t-test is increasingly non-significant until the point at which the adjusted means become equal (where the lines intersect in Fig. 3) and then the p-value starts to decrease.
Effect of POST differences between Treatment and Control on adjusted POST score means (see text for details).
Effect of POST differences between Treatment and Control on p-values from t-test of adjusted POST score means (see text for details).
