Individual patient data (IPD) meta-analyses, that is, meta-analyses that use individual patient data rather than simply the overall results of each trial, have been proposed as a major improvement in meta-analytic methods. As IPD meta-analyses are typically based on more detailed data, they usually have greater statistical power to carry out informative subgroup analyses. Moreover, IPD meta-analyses allow more accurate classification of patients based on individual characteristics, and may, therefore, allow a more thorough assessment as to whether differences in effect estimates between clinically relevant patient subgroups are spurious or not (Reference Clarke, Stewart, Egger, Smith and Altman3;Reference Stewart and Tierney15).
Previous studies (Reference Jeng, Scott and Burmeister5;Reference Steinberg, Smith and Stroup13;Reference Stewart and Parmar14) have shown that the overall effects of an intervention obtained from individual patient data or conventional meta-analyses are in the same direction, especially when similar methods of data-analysis are applied (Reference Olkin and Sampson9). Thus far, no study has been performed that compared effects in patient subgroups estimated by conventional or individual patient data meta-analyses using identical data. We compared the subgroup effects between three meta-analytic approaches, that is, (i) conventional meta-analysis, (ii) two-stage approach IPD meta-analysis, and (iii) one-stage approach IPD meta-analysis, using the data of six trials on the effectiveness of antibiotics in children with acute otitis media (AOM).
METHODS
For this empirical study, the data of six trials were available, which were previously used for an IPD meta-analysis assessing the effectiveness of antibiotics in children with AOM. Its methods and main results are extensively described elsewhere (Reference Rovers, Glasziou and Appelman10). In brief, individual patient data on 1,643 children aged 6 months to 12 years were included. The primary outcome measure was pain or fever or both at 3 to 7 days, and age (<2 and ≥2 years) and uni/bilaterality appeared to be the clinically relevant subgrouping variables.
For the present study, the summary statistics for the conventional meta-analysis based on published data were extracted from the published reports of the six included trials. The summary statistics for each trial included in the two-stage approach to IPD meta-analysis were extracted from the available individual patient data, subsequently, the summary statistics were pooled and analyzed using conventional meta-analyses techniques (Reference Simmonds, Higgins and Stewart11). For the one-stage approach to IPD meta-analysis, the individual patient data are modeled directly as if all data belong to a single trial, including a covariate for “study” to adjust for potential study dif-ferences (Reference Simmonds, Higgins and Stewart11).
Stratified analyses were performed to quantify the effect in the subgroups studied. Relative risks (RR), risk differences (RD), and their 95 percent confidence intervals (CI) of the subgrouping effects were calculated and compared between each meta-analytic approach.
RESULTS
Only two of the six included studies reported subgroup effects according to age in the trial publications; none of these reported on subgroup effects for uni- and bilateral AOM. Thus, only the results of the subgroup age could be compared between the conventional and the one- and two-stage approach to IPD meta-analyses, whereas for both IPD meta-analyses the subgroup effects of age and bilaterality could be studied. However, the two studies that could be included in the conventional meta-analyses reported end points that were different both from each other and from the outcome of the IPD meta-analyses. Appelman et al. (Reference Appelman, Claessen and Touw-Otten1) presented subgroup effects for age on an irregular course at 3 days, while McCormick et al. (Reference McCormick, Chonmaitree and Pittman8) presented subgroup effects for age regarding treatment failure between 0 and 12 days. For the IPD meta-analyses, the original primary and/or secondary outcome variables could be recoded into one similar outcome variable, notably, having ear pain, fever, or both at 3–7 days.
Table 1 displays the effect estimates (RR and RD) with their 95 percent CI of the three types of meta-analyses. The conventional meta-analysis (CMA) showed larger treatment effects for the subgroup age < 2 years than both IPD meta-analyses (IPDMA-1 and IPDMA-2), which showed similar treatment effects (RDCMA -21 percent, RDIPDMA-1 -16 percent, RDIPDMA-2 -15 percent; RRCMA 0.33, RRIPDMA-1 0.67, RRIPDMA-2 0.68; Table 1). For the subgroup age ≥2 years, the RD of the conventional meta-analysis showed a smaller effect as compared to the one- and two-stage approach IPD meta-analyses, while both IPD meta-analyses showed similar treatment effects (RDCMA -5 percent, RDIPDMA-1 -11 percent, RDIPDMA-2 -11 percent; Table 1). The RRs for the subgroup age ≥2 years were comparable between the three meta-analytic approaches (RRCMA 0.62, RRIPDMA-1 0.63, RRIPDMA-2 0.64; Table 1). The confidence intervals of the conventional meta-analyses were, however, wider (Table 1). The treatment effects for the subgroups, in which age and bilaterality were combined, were similar for the one- and two-stage approach to IPD meta-analyses (Table 1).
Subgroup Results (Relative Risk (RR), Risk Difference (RD), and Their 95% Confidence Intervals (CI)) of the Three Meta-analytic Approaches
a The two studies included in the CMA reported different end points, while for the IPD meta-analyses, the original outcome variables were recoded into one similar outcome variable.
b 1 = Appelman et al. (1991; reference 1); 2 = Burke et al. (1991; reference 2); 3 = Damoiseaux et al. (2000; reference 4); 4 = Le Saux et al. (2005; reference 6); 5 = Little et al. (2001; reference 7); 6 = McCormick et al. (2005; reference 8).
CMA, conventional meta-analyses; IPDMA-1, individual patient data meta-analyses were the IPD is modeled directly; IPDMA-2, individual patient data meta-analyses were summary statistics are extracted from the IPD; AOM, acute otitis media.
DISCUSSION
To our knowledge, this is the first study comparing intervention effects in patient subgroups resulting from conventional meta-analysis and the one- and two-stage approach IPD meta-analyses using identical trials. Our data confirm earlier studies showing that the performance of subgroup analyses in conventional meta-analyses is hampered because most studies do not report on subgroup effects (Reference Smith, Williamson and Marson12;Reference Stewart and Tierney15). Furthermore, the conventional meta-analysis showed larger (age < 2 years) or smaller (age ≥ 2 years) subgroup effect estimates and wider confidence intervals than both IPD meta-analyses. Differences in the results of conventional and IPD meta-analyses may be due to the use of other data analytical techniques, discrepancies in outcome scales, limited availability of outcome data in subgrouping variable strata, or missing data in the subgrouping variables. More likely, however, these discrepancies can be explained by the fact that the two studies that could be included in the conventional meta-analysis reported outcomes that were defined differently, both from each other and from the outcome used in the IPD meta-analyses. For the purpose of the comparison, we pooled the different outcomes of the studies included in the conventional meta-analysis anyway, but of course in other circumstances we would probably have decided that the outcomes were too heterogeneous to pool.
In addition, fewer studies reported on effects in clinically relevant patient subgroups, reducing the precision of subgroup effect estimates from the conventional meta-analysis. It should, however, be noted that lack of reporting subgroup effects does not mean that this subgroup information is not available. The IPD actually showed that almost all subgrouping variables were measured in all trials.
We did not find differences in subgroup effects between the one- and two-stage approach IPD meta-analyses, in our example. Most likely this is a result of the rather straightforward analytical techniques that were used. The two-stage approach, that is, analyzing each trial separately using its IPD before the summary results from each trial are pooled and analyzed using conventional meta-analyses techniques, has been recommended to prevent confounding (Reference Simmonds, Higgins and Stewart11;Reference Stewart and Tierney15). The one-stage approach in analyzing IPD meta-analyses, that is, direct pooling of the IPD, however, allows more flexibility in more complex situations without loss of power due to stratification by trial (Reference Stewart and Tierney15). Furthermore, by adding a variable “study” in fixed effect regression analyses, it is also possible to adjust for potential confounding.
To overcome the problem of nonreporting of subgroup effects in trial reports, investigators might ask original trialists for stratified analyses for the subgroups of interest, which is comparable with the two-stage approach in analyzing IPD meta-analyses. However, in that case, one could probably better ask for the IPD, because IPD offer the opportunity to recode variables, update follow-up data, impute the missing data, and allows flexible analyses, and more advanced modeling techniques (Reference Stewart and Tierney15).
In conclusion, our data confirm that conventional meta-analyses do not allow proper subgroup analyses, whereas both one- and two-stage approach IPD meta-analyses allow for accurate and precise subgroup effect analyses. We found no differences between the one- and two-stage approaches to IPD meta-analyses in our empirical example.
CONTACT INFORMATION
Laura Koopman, PhD (l.koopman@umcutrecht.nl), Epidemiologist and Jurinor Researcher, Geert J. M. G. van der Heijden, PhD (g.vanderheijden@umcutrecht.nl), Associate Professor of Clinical Epidemiology, Arno W. Hoes, MD, PhD (a.w.hoes@umcutrecht.nl), Professor of Clinical Epidemiology & General Practice, Diederick E. Grobbee, M.D., Ph.D. (d.e.grobbee@umcutrecht.nl), Professor of Clinical Epidemiology and Chairman, Julius Center, Maroeska M. Rovers, PhD, (m.rovers@umcutrecht.nl), Clinical Epidemiologist, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Heidelberglaan 100, Utrecht, 3584, The Netherlands.
