Hostname: page-component-7b7567c8c-7pwfn Total loading time: 0 Render date: 2026-11-16T16:56:31.770Z Has data issue: false hasContentIssue false

Developing an approach for assigning GRADE levels in a systematic overview of reviews of diagnostic test accuracy using general principles identified from current GRADE guidelines: A case study

Published online by Cambridge University Press:  13 October 2025

Andrew Dullea*
Affiliation:
Discipline of Public Health and Primary Care, School of Medicine, Trinity College Dublin, The University of Dublin, Dublin, IrelandHealth Technology Assessment Directorate, Health Information and Quality Authority , Cork, Ireland
Lydia O’Sullivan
Affiliation:
Health Technology Assessment Directorate, Health Information and Quality Authority , Cork, IrelandHealth Research Board-Trials Methodology Research Network, College of Medicine, Nursing and Health Sciences, University of Galway , Galway, Ireland
Kirsty K. O’Brien
Affiliation:
Health Technology Assessment Directorate, Health Information and Quality Authority , Cork, Ireland
Patricia Harrington
Affiliation:
Health Technology Assessment Directorate, Health Information and Quality Authority , Cork, Ireland
Marie Carrigan
Affiliation:
Health Technology Assessment Directorate, Health Information and Quality Authority , Cork, Ireland
Susan Ahern
Affiliation:
Health Technology Assessment Directorate, Health Information and Quality Authority , Cork, Ireland
Maeve McGarry
Affiliation:
Health Technology Assessment Directorate, Health Information and Quality Authority , Cork, Ireland
Karen Cardwell
Affiliation:
Health Technology Assessment Directorate, Health Information and Quality Authority , Cork, Ireland
Michelle O’Neill
Affiliation:
Health Technology Assessment Directorate, Health Information and Quality Authority , Cork, Ireland
Kieran A. Walsh
Affiliation:
Health Technology Assessment Directorate, Health Information and Quality Authority , Cork, IrelandSchool of Pharmacy, University College Cork , Cork, Ireland
Barbara Clyne
Affiliation:
Health Technology Assessment Directorate, Health Information and Quality Authority , Cork, IrelandDepartment of Public Health and Epidemiology, Royal College of Surgeons in Ireland (RCSI), University of Medicine and Health Sciences, Dublin, Ireland
Susan M. Smith
Affiliation:
Discipline of Public Health and Primary Care, School of Medicine, Trinity College Dublin, The University of Dublin, Dublin, Ireland
Mairin Ryan
Affiliation:
Health Technology Assessment Directorate, Health Information and Quality Authority , Cork, IrelandDepartment of Pharmacology and Therapeutics, Trinity College Dublin, The University of Dublin, Dublin, Ireland Permanent Address of A.D.: Health Information and Quality Authority, George’s Court, George’s Lane, Smithfield, Dublin 7, D07 E98Y, Ireland.
*
Corresponding author: Andrew Dullea; Email: dulleaa@tcd.ie
Rights & Permissions [Opens in a new window]

Abstract

Existing guidelines on overviews of reviews and umbrella reviews recommend an assessment of the certainty of evidence, but provide limited guidance on ‘how to’ apply the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) to such a complex evidence synthesis. We share our experience of developing a ‘general principles’ approach to applying GRADE to a complex overview of reviews. The approach was developed in an iterative and exploratory manner during the planning and conduct of an overview of reviews of a novel molecular imaging technique for the staging of prostate cancer, involving a formal review by a group of 11 methodologists/health services researchers. This approach was developed during the evidence synthesis process, piloted, and then applied to our ongoing overview of reviews. A ‘general principles’ approach of applying the domains of GRADE to an overview of reviews and arriving at an overall summary judgement for each outcome is presented. Our approach details additional factors to consider, including addressing both the primary study risk of bias as assessed by the included reviews and the risk of bias of the systematic reviews themselves, as well as the statistical heterogeneity observed in meta-analyses conducted within the included reviews. Our approach distilled key principles from the relevant GRADE guidelines and allowed us to apply GRADE to a complex body of evidence in a consistent and transparent way. The approach taken and the methods used to develop our approach may inform researchers working on overviews of reviews, umbrella reviews, or future methodological guidelines.

Information

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Open Practices
Open data
Copyright
© The Author(s), 2025. Published by Cambridge University Press on behalf of The Society for Research Synthesis Methodology
Figure 0

Figure 1Schematic depicting the development of the GRADE approach.

Figure 1

Table 1Example applying GRADE to an overview of reviews previously published in seminars of nuclear medicine.