Hostname: page-component-7b7567c8c-7pwfn Total loading time: 0 Render date: 2026-11-14T03:53:01.255Z Has data issue: false hasContentIssue false
  • English
  • Français

Causal Inference for Meta-Analysis and Multi-Level Data Structures, with Application to Randomized Studies of Vioxx

Published online by Cambridge University Press:  01 January 2025

Michael Sobel,
David Madigan and
Wei Wang
Michael Sobel*
Affiliation:
Columbia University
David Madigan
Affiliation:
Columbia University
Wei Wang
Affiliation:
Philips Research North America
*
Correspondence should be made to Michael Sobel, Department of Statistics, Columbia University, New York, NY, USA. Email: michael@stat.columbia.edu
Get accessRights & Permissions [Opens in a new window]

Abstract

We construct a framework for meta-analysis and other multi-level data structures that codifies the sources of heterogeneity between studies or settings in treatment effects and examines their implications for analyses. The key idea is to consider, for each of the treatments under investigation, the subject’s potential outcome in each study or setting were he to receive that treatment. We consider four sources of heterogeneity: (1) response inconsistency, whereby a subject’s response to a given treatment would vary across different studies or settings, (2) the grouping of nonequivalent treatments, where two or more treatments are grouped and treated as a single treatment under the incorrect assumption that a subject’s responses to the different treatments would be identical, (3) nonignorable treatment assignment, and (4) response-related variability in the composition of subjects in different studies or settings. We then examine how these sources affect heterogeneity/homogeneity of conditional and unconditional treatment effects. To illustrate the utility of our approach, we re-analyze individual participant data from 29 randomized placebo-controlled studies on the cardiovascular risk of Vioxx, a Cox-2 selective nonsteroidal anti-inflammatory drug approved by the FDA in 1999 for the management of pain and withdrawn from the market in 2004.

Keywords

causal inferenceindividual participant datameta-analysismulti-level modelsrandomized experimentresearch synthesis

Information

Type
Original Paper
Information
Psychometrika , Volume 82 , Issue 2 , June 2017, pp. 459 - 474
Copyright
Copyright © 2016 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

Electronic supplementary material The online version of this article (doi:10.1007/s11336-016-9507-z) contains supplementary material, which is available to authorized users.

For helpful comments on a previous draft of this paper, we are grateful to Eva Petkova, Yajuan Si, three anonymous reviewers, and the editor. Sobel’s research was supported by NIH grant R01EB016061.

References

Aitkin, M. (1999). Meta-analysis by random effect modelling in generalized linear models. Statistics in Medicine, 18, (17–18), 2343–23513.0.CO;2-3>CrossRefGoogle ScholarPubMed
Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, (6), 716–723CrossRefGoogle Scholar
Bloxom, B. (1985). Considerations in psychometric modeling of response time. Psychometrika, 50, (4), 383–397CrossRefGoogle Scholar
Campbell, D. T., & Stanley, J. C., & Gage, N. L. (1963). Experimental and quasi-experimental designs for research, Boston: Houghton Mifflin.Google Scholar
Cooper, H., & Patall, E. A. (2009). The relative benefits of meta-analysis conducted with individual participant data versus aggregated data. Psychological Methods, 14, (2), 165CrossRefGoogle ScholarPubMed
Covey, J. (2007). A meta-analysis of the effects of presenting treatment benefits in different formats. Medical Decision Making, 27, (5), 638–654CrossRefGoogle ScholarPubMed
Cox, D. R. (1972). Regression models and life-tables regression models and life-tables. Journal of the Royal Statistical Society. Series B, 34, (2), 187–220.CrossRefGoogle Scholar
DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, (3), 177–188CrossRefGoogle ScholarPubMed
Duchateau, L., & Janssen, P. (2007). The frailty model, New York: Springer.Google Scholar
Goldstein, H., & Yang, M., & Omar, R., & Turner, R., & Thompson, S. (2000). Meta-analysis using multilevel models with an application to the study of class size effects. Journal of the Royal Statistical Society: Series C, 49, (3), 399–412Google Scholar
Higgins, J., & Jackson, D., & Barrett, J., & Lu, G., & Ades, A., & White, I. (2012). Consistency and Inconsistency in network meta-analysis: Concepts and models for multi-arm studies. Research Synthesis Methods, 3, (2), 98–110CrossRefGoogle ScholarPubMed
Higgins, J., & Thompson, S. G., & Spiegelhalter, D. J. (2009). A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society: Series A, 172, (1), 137–159CrossRefGoogle ScholarPubMed
Higgins, J., & Whitehead, A., & Turner, R. M., & Omar, R. Z., & Thompson, S. G. (2001). Meta-analysis of continuous outcome data from individual patients. Statistics in Medicine, 20, (15), 2219–2241CrossRefGoogle ScholarPubMed
Hong, G., & Raudenbush, S. W. (2006). Evaluating kindergarten retention policy. Journal of the American Statistical Association, 101, (475), 901–910CrossRefGoogle Scholar
Imbens, G. W., & Rubin, D. B. (2015). Causal Inference in statistics, social, and biomedical sciences, New York: Cambridge University PressCrossRefGoogle Scholar
Jüni, P., & Nartey, L., & Reichenbach, S., & Sterchi, R., & Dieppe, P. A., & Egger, M. (2004). Risk of cardiovascular events and rofecoxib: Cumulative meta-analysis. The Lancet, 364, (9450), 2021–2029CrossRefGoogle ScholarPubMed
Kalbfleisch, J. D., & Prentice, R. L. (2002). The statistical analysis of failure time data, 2Hoboken, NJ: WileyCrossRefGoogle Scholar
Kearney, P. M., & Baigent, C., & Godwin, J., & Halls, H., & Emberson, J. R., & Patrono, C. (2006). Do selective cyclo-oxygenase-2 inhibitors and traditional non-steroidal anti-inflammatory drugs increase the Risk of Atherothrombosis?. Meta-analysis of Randomised Trials British Medical Journal, 332, (7553), 1302–1308Google ScholarPubMed
Kivimäki, M., Nyberg, S. T., Batty, G. D., Fransson, E. I., Heikkilä, K., Alfredsson, L., \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ldots $$\end{document}, IPD-Work Consortium. (2012). Job strain as a risk factor for coronary heart disease: A collaborative meta-analysis of individual participant data. The Lancet, 380(9852), 1491–1497.Google Scholar
Konstam, M. A., & Weir, M. R., & Reicin, A., & Shapiro, D., & Sperling, R. S., & Barr, E. (2001). et al. Cardiovascular thrombotic events in controlled, clinical trials of rofecoxib. Circulation, 104, (19), 2280–2288CrossRefGoogle ScholarPubMed
Landoni, G., & Greco, T., & Biondi-Zoccai, G., & Neto, C. N., & Febres, D., & Pintaudi, M. (2013). et al. Anaesthetic drugs and survival: A Bayesian network meta-analysis of randomized trials in cardiac surgery. British Journal of Anaesthesia, 111, (6), 886–896CrossRefGoogle ScholarPubMed
Lumley, T. (2002). Network meta-analysis for indirect treatment comparisons. Statistics in Medicine, 21, (16), 2313–2324CrossRefGoogle ScholarPubMed
Petkova, E., & Tarpey, T., & Huang, L., & Deng, L. (2013). Interpreting meta-regression: Application to recent controversies in antidepressants efficacy. Statistics in Medicine, 32, (17), 2875–2892CrossRefGoogle ScholarPubMed
Raudenbush, S. W. Cooper, H., & Hedges, L. V., & Valentine, J. C. (2009). Analyzing effect sizes: Random-effects models. The handbook of research synthesis and meta-analysis, New York: Russell Sage Foundation 295–316.Google Scholar
Reicin, A. S., & Shapiro, D., & Sperling, R. S., & Barr, E., & Yu, Q. (2002). Comparison of cardiovascular thrombotic events in patients with osteoarthritis treated with rofecoxib versus nonselective nonsteroidal anti-inflammatory drugs (ibuprofen, diclofenac, and nabumetone). The American Journal of Cardiology, 89, (2), 204–209CrossRefGoogle ScholarPubMed
Rosenbaum, P. R. (1989). The role of known effects in observational studies. Biometrics, 45, 557–569CrossRefGoogle Scholar
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, (1), 41–55CrossRefGoogle Scholar
Ross, J. S., & Madigan, D., & Hill, K. P., & Egilman, D. S., & Wang, Y., & Krumholz, H. M. (2009). Pooled analysis of rofecoxib placebo-controlled clinical trial data: Lessons for postmarket pharmaceutical safety surveillance. Archives of Internal Medicine, 169, (21), 1976–1985CrossRefGoogle ScholarPubMed
Rubin, D. B. (1980). Randomization analysis of experimental data–The Fisher randomization test–Comment. Journal of the American Statistical Association, 75, (371), 591–593.Google Scholar
Schoenfeld, D. (1982). Partial residuals for the proportional hazards regression model. Biometrika, 69, (1), 239–241CrossRefGoogle Scholar
Simmonds, M. C., & Higgins, J. P., & Stewart, L. A., & Tierney, J. F., & Clarke, M. J., & Thompson, S. G. (2005). Meta-analysis of individual patient data from randomized trials: A review of methods used in practice. Clinical Trials, 2, (3), 209–217CrossRefGoogle ScholarPubMed
Sobel, M. E. (2006). What do randomized studies of housing mobility demonstrate? Causal inference in the face of interference. Journal of the American Statistical Association, 101, (476), 1398–1407CrossRefGoogle Scholar
Thase, M. E., & Haight, B. R., & Richard, N., & Rockett, C. B., & Mitton, M., & Modell, J. G. (2005). et al. Remission rates following antidepressant therapy with bupropion or selective serotonin reuptake inhibitors: a meta-analysis of original data from 7 randomized controlled trials. The Journal of Clinical Psychiatry, 66, (8), 974–981CrossRefGoogle ScholarPubMed
Therneau, T. M. (2013). A Package for Survival Analysis in S. Retrieved from http://CRAN.R-project.org/package=survival (package version 2.37-4).Google Scholar
Tudur Smith, C., & Williamson, P. R., & Marson, A. G. (2005). Investigating heterogeneity in an individual patient data meta-analysis of time to event outcomes. Statistics in Medicine, 24, (9), 1307–1319CrossRefGoogle Scholar
Weir, M. R., & Sperling, R. S., & Reicin, A., & Gertz, B. J. (2003). Selective COX-2 inhibition and cardiovascular effects: A review of the rofecoxib development program. The American Heart Journal, 146, (4), 591–604CrossRefGoogle ScholarPubMed
Zhang, J., & Ding, E. L., & Song, Y. (2006). Adverse effects of cyclooxygenase 2 Inhibitors on renal and arrhythmia events. Journal of the American Medical Association, 296, (13), 1619–1632CrossRefGoogle ScholarPubMed
Supplementary material: File

Sobel et al. supplementary material

Sobel et al. supplementary material
Download Sobel et al. supplementary material(File)
File 2 KB