Knowledge synthesis involves bringing together findings from individual research studies to answer a specific question or explore a particular topic, often at a global level. People with lived or living experience of health conditions or of accessing healthcare services (PWLE), such as patients and their families or friends, can provide valuable perspectives throughout this process. PWLE involvement can help shape the design and conduct of a knowledge synthesis to better reflect real-world concerns through a process called patient engagement. Patient engagement is the meaningful and active involvement of PWLE throughout the research process. While previous research shows that PWLE can contribute meaningfully when appropriate supports are in place, there is limited practical guidance on how to enact patient engagement in knowledge syntheses. This tutorial paper aims to guide researchers and PWLE who would like to engage in knowledge syntheses together. Building on existing research, our team (comprised of PWLE, researchers, and an academic librarian) shares lessons we have learned from working together throughout the stages of a knowledge synthesis. We present considerations for successful patient engagement when research teams are: (1) planning to engage, (2) recruiting PWLE, (3) establishing roles and rapport, (4) supporting capacity development, (5) conducting the knowledge synthesis, and (6) mobilizing findings. We also identify 12 common barriers to patient engagement and offer strategies to address them, alongside recommendations for integrating PWLE across every stage of the knowledge synthesis process.

Flexible meta-regression is an important facet of biomedical research. These methods employ basis expansions to model the relationship between exposure and outcome. Examples include splines, fractional polynomial, and local regression models. The extent to which flexible meta-regression has been appropriately employed and reported in the past has not been studied. This study aims to provide an overview of flexible meta-regression for researchers as well as summarize the practical use and reporting of these methods in the epidemiological literature. A systematic literature search of EMBASE, MEDLINE, and the Cochrane Library identified systematic reviews published between 2011 and 2021 which employed flexible meta-regression methods. Data on study characteristics were extracted and validated by two independent researchers. Given the lack of guidance on reporting standards for flexible meta-regression, we proposed a novel 9-item checklist. This was used to assess reporting quality in included studies. A total of $N=346$ eligible studies were identified and $N=86$ were randomly sampled for full data extraction and analysis. The number of reviews using flexible regression methods increased over time. Spline models were the most frequently employed class of methods. There were no apparent trends in the use of the three methods over country of affiliation, number of authors, or clinical context. In over two thirds of reviews sampled, methods were found to have not been reported consistently. Further work is needed to improve the reporting of these methods to ensure research transparency and reproducibility.

Scoping reviews are increasingly used for evidence synthesis, yet a comprehensive overview of benchmarks for their conduct is lacking. This study aimed to extract and analyze key metrics of preregistered scoping reviews to aid researchers in planning, resource allocation, methodological rigor, and innovation. We examined 2,038 scoping review protocol registrations on the Open Science Framework between April and August 2024, identifying 891 corresponding publications. Extracted variables included review type (e.g., rapid scoping review), information about the number of studies in the screening process, methodological practices, and funding. Using Stata, we calculated process metrics such as review duration, yield rate, and team composition. Qualitative analysis was applied to further metrics, including aims and eligibility criteria frameworks. Among the 891 publications, 91.47% were published as scoping reviews. On average, reviews involved 6.11 authors and took 80.41 weeks from registration to publication, with rapid scoping reviews completed in 55.63 weeks. Most publications (80.13%) originated from high-income countries, where funding was more common. An average of 5.19 databases were searched, identifying a mean of 6,920.66 potential inclusions, with 92.64 studies ultimately included (yield rate: 5.68%). Endnote was the most frequently employed screening tool. Language restrictions were reported in 37.60% of cases, and 75.42% did not conduct or report critical appraisal. This study provides valuable insights into common practices within scoping review processes, offering researchers a foundation to allocate resources more efficiently and enhance research quality through methodological benchmarks. It underscores the need for greater transparency and improved methodological rigor.

This study investigated information retrieval of preprint records in the context of evidence synthesis work and compared 12 sources used to discover preprints. Identification of grey literature is often required or recommended in evidence synthesis guidance, and preprints are categorized as grey literature. The purpose of this work is to inform how and where to search for preprints to maximize coverage (through exploration of preprint server across aggregators and databases) while balancing search efficiency. Authors selected aggregators and databases hosting two or more preprint servers and then tested search functionality and extracted characteristics and features. Authors analyzed and compared the selected sources, tabulated the number of essential features, and created comparison tables reflecting database and aggregator features. The study protocol was registered in Open Science Framework registries. Preprint aggregators and databases differ in their content coverage, and their ability to design a comprehensive and reproducible search strategy. Limitations such as character or word limits for queries, limited advanced search operators, and missing export functionality affect the usability of aggregators for evidence synthesis searches. Ongoing updates to search interfaces and functionality and differing approaches to versioning make it challenging to study discovery of preprints across sources. The recommendations and scenarios in this article will assist searchers engaged in evidence synthesis to make informed decisions about where to search for preprints.

A key task in conducting systematic reviews is deduplicating the results from database searching. Deduplication using reference management software can be time-consuming and prone to error, while automated tools can be expensive and lack transparency. To support review teams, we evaluated eight deduplication tools: (1) The Automated Systematic Search Deduplicator (ASySD); (2) Covidence; (3) Deduklick; (4) EPPI-Reviewer; (5) PICO Portal; (6) Rayyan; (7) The Systematic Review Accelerator (SRA) Deduplicator: Focused; (8) The SRA Deduplicator: Relaxed. Five randomly selected Cochrane reviews had their searches rerun to create five gold standard sets. We compared the gold standard sets to the outputs of the eight deduplication tools and evaluated the results for: (1) unique records removed; (2) duplicate records retained; (3) time taken to deduplicate. Summed across all five reviews, the unique records removed in error ranged from 2 to 22. The three best tools were: (1) Rayyan; (2) Covidence; (3) SRA Deduplicator: Focused. The duplicate records retained in error ranged from 34 to 280, the three best tools were: (1) ASySD; (2) Rayyan; (3) EPPI-Reviewer. The time taken to deduplicate ranged from one minute to 20 hours and 34 minutes, the three fastest tools were: (1) SRA Deduplicator: Relaxed; (2) Deduklick; (3) Covidence. No tool performed so poorly that we don’t recommend using it. But, as all the tools had strengths and weaknesses, some are expensive while others require large amounts of manual checking time, we recommend review teams compare the tools across all three outcomes and choose the tool that best suits their needs.

Individual patient data meta-analyses (IPDMAs) provide powerful tools for synthesizing evidence across studies, yet methods for addressing unmeasured confounding in observational IPDMAs with survival outcomes are rarely implemented. Instrumental variable (IV) approaches offer causal inference capabilities but face practical challenges in hierarchical data structures, particularly the lack of standard diagnostics for instrument strength in nonlinear mixed-effects models. We adapt and evaluate a frequentist mixed-effects two-stage residual inclusion (2SRI) framework for survival IPDMAs, extending traditional IV methods to accommodate study-level and temporal clustering while handling time-to-event outcomes through Cox proportional hazards models. Because classical F-statistics are unavailable for logistic mixed-effects first-stage models, we propose the Wald $\chi ^2$ statistic as a practical instrument-strength diagnostic and empirically characterize its relationship to estimator performance. Through a comprehensive simulation study with 48 scenarios—varying unmeasured confounding (weak to very strong), instrument–treatment association strength (0.3–1.0), and cross-study IV allocation patterns—we evaluated 2SRI against naive mixed-effects Cox models using bias, coverage, variance, and mean squared error. The design was anchored to realistic IPDMA structure (10 studies, $N \approx 4,357$) from pooled Ebola data, with 1,000 replications per scenario. Results show that under weak confounding, naive models dominate on all metrics. With moderate-to-strong confounding and realized Wald $\chi ^2$ exceeding 150–200, mixed-effects 2SRI substantially reduces bias and achieves near-nominal coverage, though with inflated variance. We provide empirical guideposts linking realized first-stage strength to expected performance, enabling analysts to judge when 2SRI will outperform conventional approaches in hierarchical survival IPDMAs. All simulations assume a common treatment effect across studies. Performance under heterogeneous effects remains to be established.

Externally controlled single-arm trials are critical to assess treatment efficacy across therapeutic indications for which randomized controlled trials are not feasible. A closely-related research design, the unanchored indirect treatment comparison, is often required for disconnected treatment networks in health technology assessment. We present a unified causal inference framework for both research designs. We develop an estimator that augments a popular weighting approach based on entropy balancing—matching-adjusted indirect comparison (MAIC)—by fitting a model for the conditional outcome expectation. The predictions of the outcome model are combined with the entropy balancing MAIC weights. While the standard MAIC estimator is singly robust where the outcome model is non-linear, our augmented MAIC approach is doubly robust (DR), providing increased robustness against model misspecification. This is demonstrated in a simulation study with binary outcomes and a logistic outcome model, where the augmented estimator demonstrates its DR property, while exhibiting higher precision than all non-augmented weighting estimators and near-identical precision to G-computation. We describe the extension of our estimator to the setting with unavailable individual participant data for the external control, illustrating it through an applied example. Our findings reinforce the understanding that entropy balancing-based approaches have desirable properties compared to standard “modeling” approaches to weighting, but should be augmented to improve protection against bias and guarantee double robustness.

To examine the extent to which information sources other than journal articles are sought for systematic reviews. Cross-sectional study of published systematic reviews. We examined all published systematic reviews included in MEDLINE in a 4-week period in 2019. Both systematic reviews and protocols of reviews were eligible for inclusion. (1) Number and types of information sources sought in systematic reviews; (2) proportion of reviews that explicitly searched for study reports other than journal articles; (3) proportion of reviews that searched resources containing study reports other than journal articles. A total of 1,262 systematic reviews fulfilled the eligibility criteria. The median number of information resources searched for all systematic reviews was 4. Of the 1,262 reviews, study reports other than journal articles were sought in 40% (n = 502) of systematic reviews (97% (n = 64) of Cochrane reviews and 37% (n = 438) of non-Cochrane reviews). Trial registers were searched in 88% of Cochrane reviews and 21% of non-Cochrane reviews. In 99.3% (n = 1,253) of all the systematic reviews, the searches performed had the potential to identify study reports other than journal articles. Between a third and a half of systematic reviews search for study reports other than journal articles. Systematic review searches often search resources that include study reports other than journal articles, whether or not the reviewers explicitly sought them.

To explore the perceptions of, and barriers to, grey literature searching among medical researchers and journal editors. A cross-sectional survey of authors of systematic reviews and the editors of the journals in which the reviews were published. Systematic reviews indexed in MEDLINE, spanning a 4-week period in 2019. We excluded protocols. We asked whether the reviewers were performing a grey literature search. If they were, we asked about their approach to the grey literature search and relevant guidance. If they were not, we asked about their rationale for this. We elucidated understandings of grey literature from all reviewers. The survey to journal editors asked about their perceptions towards grey literature. A consecutive sample of 1,229 systematic reviews was included. A total of 155 authors responded, and a total of 46 journal editors responded. The majority (57%) of reviewers reported performing a grey literature search. However, there was no consensus on types of grey literature items or sources among the reviewers. The most frequent barrier to grey literature searches was concern about the quality/detail of the grey literature items. Editors expressed negative perceptions towards grey literature, rooted in suspicion of its quality. While many reviewers reported performing a grey literature search, there was a diverse understanding of the term grey literature. Concerns about the quality of grey literature exist among reviewers and editors alike. There is a marked discrepancy between the best-practice guidelines and the gatekeepers of journals.

Categorical moderators are often found in meta-analysis and examined using meta-regression models. When multiple effect sizes are present within studies, several methods can be used for meta-regression: multivariate models, three-level models, correlated-effects models with robust variance estimation (RVE), three-level models with RVE, and correlated-effects models with RVE and cluster wild bootstrapping (CWB). This study aimed to compare the performance of these methods through a simulation study. Cohen’s d values were generated under a multivariate model, incorporating a binary variable that could represent either study-level or effect size-level characteristics. When the moderator referred to an effect size-level characteristic, its effect was allowed to vary across studies. Factors manipulated in the simulation included number of studies, number of outcomes per study, and the distribution of effect sizes across the categories of the moderator variable, ranging from balanced to highly unbalanced. The methods were applied and compared in terms of bias, Type I error, and power. The results showed that all methods exhibited lower power to detect effects when the moderator variable referred to study-level characteristics and the effect size distribution was very unbalanced. Methods based on RVE (correlated-effects with RVE or with RVE and CWB, and three-level models with RVE) effectively controlled Type I error rates but tended to be overconservative. In contrast, three-level models achieved higher power but at the cost of inflated Type I error. The best balance between Type I error control and power was observed when using a combination of three-level models and RVE.

In meta-analyses, effect size measures with bounded or non-normal sampling distributions are commonly analyzed on a transformed scale to justify normality assumptions. While point estimates and confidence intervals (CIs) are routinely back-transformed to the original scale for interpretation, this practice is nontrivial in random-effects models. In particular, standard inverse back-transformations yield estimates of the median rather than the mean effect size due to Jensen’s inequality. Integral back-transformations provide a principled solution for recovering the mean on the original scale, but their use entails practical issues. We study integral back-transformations for several effect size measures, including correlation coefficients, proportions, odds and risk ratios, and Cronbach’s alpha. We derive general formulations for integral back-transformations and corresponding CIs that are applicable across different transformation functions and provide a software implementation. Although required to obtain correct mean estimates, these approaches must be used with caution, as they are sensitive to heterogeneity estimation and can be unstable for unbounded transformations. Certain asymmetric transformations can also lead to inconsistent inference results. We illustrate these issues using analytical considerations and re-analyses of several meta-analyses. Importantly, we stress that the choice of back-transformation depends on the analyst’s goals. The standard inverse back-transformation remains well suited for descriptive purposes and is often preferable in practice, provided that it is correctly interpreted as a back-transformed median effect size. We recommend using the integral back-transformation only when a mean estimate is explicitly required, and restricting its use for CIs to cases where they are demanded, but not for hypothesis testing.

Meta-analysis is a widely used statistical tool for estimating the diagnostic accuracy of tests across multiple studies. Existing methods and available R packages primarily focus on a single diagnostic test, typically under the assumption that all studies include a gold standard. Greater efficiency can be achieved by modeling multiple diagnostic tests together and drawing on studies with or without a gold standard reference test across diverse designs. To address this challenge, recent work has extended both the Bayesian hierarchical model and the Bayesian hierarchical summary receiver operating characteristic model to the framework of network meta-analysis of diagnostic tests, enabling simultaneous comparison of multiple tests when some data are missing. Despite the importance of these methods, their computational complexity has limited their broad application. This article introduces NMADTA, an R package that implements these models with user-friendly functions. The package allows researchers to evaluate the accuracy of multiple diagnostic tests simultaneously and provides comprehensive graphical displays of the results.