Umbrella reviews (URs) synthesize findings from multiple systematic reviews on a specific topic. Methodological approaches for analyzing and presenting UR results vary, and reviewers often adapt methods to align with research objectives. This study examined the characteristics of analysis and presentation methods used in healthcare-related URs. A systematic PubMed search identified URs published between 2023 and 2024. Inclusion criteria focused on healthcare URs using systematic reviews as the unit of analysis. A random sample of 100 eligible URs was included. A customized, piloted data extraction form was used to collect bibliographic, conduct, and reporting data independently. Descriptive analysis and narrative synthesis summarized findings. The most common terminology for eligible studies was “umbrella reviews” (65%) or “overviews” (30%). Question frameworks included PICO (43%) and PICOS (14%), with quantitative systematic reviews included in most URs (98%), and 68% including randomized controlled trials. The most frequent methodological guidance source was Cochrane (32%). Data analysis commonly used narrative synthesis and meta-analysis, with Stata, RevMan, and GRADEPro GDT employed for presentation. Information about study overlap and certainty assessment was rarely reported.Variation exists in how data are analyzed and presented in URs, with key elements often omitted. These findings highlight the need for clearer methodological guidance to enhance consistency and reporting in future URs.

The ctrdata package has been created to boost the use of data available in public registers of clinical trials. It enables user-friendly, reproducible workflows to identify trials of interest, download protocol- and results-related data, and conduct sophisticated analyses, across multiple registers and trials. ctrdata works in the widely used R environment, and its databases can be used with other tools. The package is open source with a permissive licence, to facilitate collaboration.

This report provides an overview of ctrdata, including its implementation, cases of interest to researchers in public health, medicines, and regulatory science, as well as potential limitations and further developments. At this time, ctrdata works with the European Union (EU) Clinical Trials Information System (CTIS), the EU Clinical Trials Register (EUCTR), the US Clinicaltrials.Gov (CTGOV), and the ISRCTN—the UK’s Clinical Study Registry. The registers are complementary in scope and scientific value, yet differences in data models, variable definitions, search parametrisations, and retrieval options hamper efficient scientific workflows, calling for a scientific-technical, programmatic solution and driving the development of ctrdata.

By employing ctrdata to comprehensively use and easily leverage trial register data, researchers can effectively address a variety of questions, gain useful insights into evolving policies and practices of drug development, and inform further clinical research. Patients and their organisations, developers, policymakers, and other interested parties can build on ctrdata to create solutions for their use cases.

In random-effects meta-analysis, the between-study heterogeneity variance, $\tau ^2$, is often reported but is not easy to interpret. For meta-analyses of differences (such as mean differences, standardized mean differences, or risk differences), the standard deviation (SD), $\tau $, indicates the extent to which studies’ true effects vary about their average. For meta-analyses of (natural) log-transformed measures of effect (such as log risk ratios [RRs]), we explain how the geometric SD, $\exp (\tau )$, is helpful to understand how untransformed measures (such as RRs) vary multiplicatively about their average. We recommend that authors and software developers report $\tau $ for differences and $\exp (\tau )$ for ratios, rather than $\tau ^2$. This will facilitate the interpretation of the magnitude of heterogeneity values, for example, the interpretation of heterogeneity estimates and confidence intervals beyond simple binary statements about the presence or absence of heterogeneity.

In meta-analyses, effect size measures with bounded or non-normal sampling distributions are commonly analyzed on a transformed scale to justify normality assumptions. While point estimates and confidence intervals (CIs) are routinely back-transformed to the original scale for interpretation, this practice is nontrivial in random-effects models. In particular, standard inverse back-transformations yield estimates of the median rather than the mean effect size due to Jensen’s inequality. Integral back-transformations provide a principled solution for recovering the mean on the original scale, but their use entails practical issues. We study integral back-transformations for several effect size measures, including correlation coefficients, proportions, odds and risk ratios, and Cronbach’s alpha. We derive general formulations for integral back-transformations and corresponding CIs that are applicable across different transformation functions and provide a software implementation. Although required to obtain correct mean estimates, these approaches must be used with caution, as they are sensitive to heterogeneity estimation and can be unstable for unbounded transformations. Certain asymmetric transformations can also lead to inconsistent inference results. We illustrate these issues using analytical considerations and re-analyses of several meta-analyses. Importantly, we stress that the choice of back-transformation depends on the analyst’s goals. The standard inverse back-transformation remains well suited for descriptive purposes and is often preferable in practice, provided that it is correctly interpreted as a back-transformed median effect size. We recommend using the integral back-transformation only when a mean estimate is explicitly required, and restricting its use for CIs to cases where they are demanded, but not for hypothesis testing.

Meta-analysis is a widely used statistical tool for estimating the diagnostic accuracy of tests across multiple studies. Existing methods and available R packages primarily focus on a single diagnostic test, typically under the assumption that all studies include a gold standard. Greater efficiency can be achieved by modeling multiple diagnostic tests together and drawing on studies with or without a gold standard reference test across diverse designs. To address this challenge, recent work has extended both the Bayesian hierarchical model and the Bayesian hierarchical summary receiver operating characteristic model to the framework of network meta-analysis of diagnostic tests, enabling simultaneous comparison of multiple tests when some data are missing. Despite the importance of these methods, their computational complexity has limited their broad application. This article introduces NMADTA, an R package that implements these models with user-friendly functions. The package allows researchers to evaluate the accuracy of multiple diagnostic tests simultaneously and provides comprehensive graphical displays of the results.

Existing guidelines on overviews of reviews and umbrella reviews recommend an assessment of the certainty of evidence, but provide limited guidance on ‘how to’ apply the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) to such a complex evidence synthesis. We share our experience of developing a ‘general principles’ approach to applying GRADE to a complex overview of reviews. The approach was developed in an iterative and exploratory manner during the planning and conduct of an overview of reviews of a novel molecular imaging technique for the staging of prostate cancer, involving a formal review by a group of 11 methodologists/health services researchers. This approach was developed during the evidence synthesis process, piloted, and then applied to our ongoing overview of reviews. A ‘general principles’ approach of applying the domains of GRADE to an overview of reviews and arriving at an overall summary judgement for each outcome is presented. Our approach details additional factors to consider, including addressing both the primary study risk of bias as assessed by the included reviews and the risk of bias of the systematic reviews themselves, as well as the statistical heterogeneity observed in meta-analyses conducted within the included reviews. Our approach distilled key principles from the relevant GRADE guidelines and allowed us to apply GRADE to a complex body of evidence in a consistent and transparent way. The approach taken and the methods used to develop our approach may inform researchers working on overviews of reviews, umbrella reviews, or future methodological guidelines.

Evidence synthesis findings hinge upon well-designed, effective search strategies. When developing these strategies, evidence synthesis teams make multiple decisions (e.g., selecting information sources, developing search string architecture, and picking supplementary search methods) that directly affect the breadth of discovered evidence and thus evidence synthesis outcomes. Despite the number of decisions required when developing search strategies, limited guidance exists to inform these decisions using a data-driven approach. To help address this gap, we developed CiteSource, an R package and accompanying Shiny application, that supports data-driven search strategy development and reporting. CiteSource allows users to assign and retain metadata across three custom fields: source, label, and string to indicate where the records were found, what method or string was used to find them, and whether they were included after screening. CiteSource allows users to visually map the overlap between sets of records, create data summaries of citation records, and export citation records with the newly assigned metadata. CiteSource’s analysis and visualization outputs can be harnessed for a variety of use cases, such as optimizing literature source selection, honing and understanding the effectiveness of search strings, and evaluating the impacts of literature sources and supplementary search methods. Overall, CiteSource provides a tool for evidence synthesizers to make informed data-driven decisions that boost the efficiency, rigor, and transparency of search strategies and associated reporting.

Network meta-analysis (NMA) is a vital methodology for synthesizing evidence across multiple treatments and informing medical decision-making. However, effective visualization and interpretation of results from large networks of interventions remain challenging, particularly for non-specialists. NMAstudio 2.0 is an innovative, interactive web application designed to address these difficulties by streamlining NMA workflows and enhancing result visualization. Developed using Python and R, NMAstudio 2.0 seamlessly integrates with established NMA frameworks. Our exemplar application of NMAstudio 2.0 using a Cochrane Review comparing several treatments for chronic plaque psoriasis demonstrates its capacity to facilitate all crucial steps of an NMA. The application features an intuitive interface for uploading data, automating analyses, generating interactive visualizations such as network diagrams, forest plots, ranking plots, and producing unique outputs like boxplots for transitivity checks and bidimensional forest plots. Most outputs are dynamically linked with the network diagram, enabling users to interactively explore evidence networks, apply advanced filtering, and highlight specific features by selecting nodes or edges within the diagram. While NMAstudio 2.0 aims to simplify NMAs, it also incorporates steps during the data upload process to mitigate the risk of producing poorly reported NMAs. NMAstudio 2.0 represents a significant step forward in improving the usability and accessibility of NMA, offering researchers a robust, versatile platform for evidence synthesis. Its integration of advanced features with an emphasis on user experience positions it as a valuable resource for enhancing decision-making and promoting evidence-based practice across diverse contexts.

Network meta-analysis (NMA) provides a powerful framework for synthesizing evidence across multiple interventions, accommodating both direct and indirect comparisons. However, effectively visualizing the complex, multidimensional results, such as effect magnitudes, uncertainty, p-values, and treatment rankings, remains a significant challenge. Outputs such as relative treatment effects, uncertainty, statistical significance, and treatment rankings are often reported separately, making it difficult for researchers and stakeholders to synthesize findings efficiently. We introduce plate plot, an innovative approach for visualizing key outcomes from NMA in a single, compact format. It enables simultaneous display of point estimates, confidence or credible intervals, significance levels, and surface under the cumulative ranking curve values, thereby facilitating clearer interpretation and communication of NMA findings. Using an example dataset, we demonstrate how the plate plot displays multiple relevant metrics to compare the efficacy and acceptability of various antidepressant interventions in a single, intuitive plot. The plate plot, generated effortlessly via the open-source nmaplateplot R package, enables users to generate customizable, publication-ready graphics with minimal programming. This tool enhances the ability to holistically evaluate and interpret complex comparative effectiveness data, supporting better-informed decision-making in research and clinical practice.