Comprehensive reviews have become second to original research as the most common category of manuscript submitted to Prehospital and Disaster Medicine. In addition, comprehensive reviews have become the most referenced category of papers published by Prehospital and Disaster Medicine. Traditionally, comprehensive reviews have been loosely formatted with an emphasis toward an objective report and discussion of findings from a general literature search. As the science of disaster and emergency medicine has developed, so have the standards for comprehensive reviews. The most compelling comprehensive reviews are those that adhere to an organized format that closely mirrors that of original research. This editorial briefly reviews the elements and format for a persuasive comprehensive review.

Reviews are common publication formats in journals, and the rapid increase in publication of reviews has resulted in a vague understanding and definition of what constitutes an appropriate review. Reviews may include different forms of material, including published peer-reviewed papers; so-called “grey literature,” which is usually available but not peer-reviewed; and societal and organizational guidelines. Even more imprecise are the various categories of review which include systematic reviews, meta-analysis, narrative review, research synthesis, scoping review, and qualitative review. Comparisons of the validity of each form of review are currently lacking and are a potential for future research.

A comprehensive review employs Methods from some of the review categories listed above. A comprehensive review uses literature review methods that are specific and thorough to avoid outcome bias. Bias in reviews is a distortion of an estimated effect or outcome that can occur from systematic errors in design of a literature search or over-reaching bias present in published material that is available for review. A valid comprehensive review is a systematic, scientifically designed review of a defined literature base that employs the rigor of original research in an effort to limit outcome bias. A properly conducted comprehensive review has the most probability of all forms of reviews to become an important source of evidence for prehospital and disaster health and medical decision making.

Table 1 lists the basic elements for an adequate comprehensive review. A most important element of a comprehensive review is the review Objective. This requires a concise and precisely stated review question similar to the statement of a study objective or hypothesis. The review Objective usually is stated at the end of an Introduction section that describes the problem that is the subject for the review and the reason that answering the review question is important. It seems unnecessary, but although the review Objective is revealed in the justification for the review described in the Introduction, the concise statement of the review Objective at the end of the Introduction alerts readers to the precise variables and application expected of the review.

Table 1

The 12 Primary Elements of a Comprehensive Review

The quality of review methods is important, and an error that must be avoided is the inadvertent lack of inclusion of a paper that should be reviewed. Even if the paper is of low quality among better quality papers, but is not included in the review process, there is suspicion among other experts in the field that the review authors are sloppy or intentionally misleading the readership. To avoid exclusion of a potential article to be reviewed, it is important to clearly define inclusion and exclusion criteria for review papers and to rigorously search the literature for all potential literature that meets inclusion criteria as defined for the review.

A common mistake in comprehensive reviews is the equal weighting of evidence presented in all the literature included in the review. This error can lead to bias and vagueness of results. One cannot say that a single, randomized controlled trial outweighs the importance of all other non-randomized, non-controlled studies found for review, but the significance of the findings in a randomized controlled trial that properly meets inclusion criteria and is properly conducted carries much more influence than other less-rigorous studies. With this concept, the ranking of validity of research methods used for each paper, and weighting and reporting the significance of each paper reviewed, is an important element for a valid comprehensive review.

Another subtle form of bias that may enter into a comprehensive review is the publishing of single data sets in multiple publications (with essentially the same outcome) or the unethical practice of double publishing (submitting the same essential manuscript in multiple journals). Each paper included in a review should be unique and redundant information discarded with an explanation why certain papers were excluded (part of exclusion criteria should be exclusion of redundant publications).

The Results and Conclusions of a review should be presented objectively and based on the quality of the literature and without over-interpretation. This brings forward the issue of a negative review, in which there is a substantial volume of literature but no conclusive consensus to allow for answering a review question. While negative reviews rarely meet the criteria for publication because there is a lack of literature to have initiated the review, a negative review that reports results on a substantial volume of literature is valuable and helps define future research hypotheses.

In summary, comprehensive reviews are rigorous scientific studies of the literature. Comprehensive reviews can be important tools for developing consensus in prehospital and disaster health and medicine.