BACKGROUND

Randomized controlled trials (RCTs) are the gold standard of research methods in clinical scientific disciplines.Reference Ioannidis, Haidich and Pappa ¹ These trials attempt to establish causality between an intervention and outcome. Although they are the best clinical design to reduce bias and confounding, RCTs are laborious and expensive.

In a traditional RCT, patient-level randomization occurs when participants are assigned to an intervention group or a control group. In healthcare epidemiology and antimicrobial stewardship (HE&AS) research, cluster randomization, in which groups such as clinics or hospitals are randomized, is often necessary. With either method, the randomization step is the key step to decreasing bias and better establishing causality. General information on RCTs is available from texts and reviews,Reference Friedman, Furberg and DeMets ^{2 –} Reference Meurer and Lewis ⁴ Consolidated Standards of Reporting Trials (CONSORT) statements,Reference Campbell, Elbourne and Altman ⁵ and a Journal of the American Medical Association Users’ Guide to the Medical Literature.Reference Dans, Dans, Guyatt and Richardson ⁶ In this paper, we focus on important and unique aspects of RCTs in HE&AS.

ADVANTAGES AND DISADVANTAGES

Randomized controlled trials have several methodological advantages (Table 1). By clearly identifying the study population, randomizing participants or groups to reduce bias, and conducting well-accepted statistical analyses, RCTs can optimally compare 2 or more approaches. Despite the numerous advantages to RCTs, several notable disadvantages are particularly relevant to HE&AS research. For example, RCTs are time and resource intensive and require extensive regulatory oversight.Reference Baily ⁷ Finally, a RCT may not be necessary for interventions with dramatic and/or rapid effects, such as the central-line–associated bloodstream infection (CLABSI) bundleReference Pronovost, Needham and Berenholtz ⁸ or may not be ethical if the variable is the indisputable “standard of care” (eg, hand hygiene compared to no hand hygiene).

TABLE 1

Advantages, Disadvantages, and Potential Pitfalls of Using Randomized Controlled Trials (RCT) in Healthcare Epidemiology and Antimicrobial Stewardship (HE&AS) Research

PITFALLS AND TIPS

Some pitfalls in RCTs can be limited through careful planning and documentation of plans prior to initiation. Pilot studies are often employed to both estimate the potential effect of an intervention and to develop tools for implementation. Determining the appropriate outcome is essential in a RCT. Outcomes must be sufficiently common to detect a difference between experimental and control groups, easy and objective to ascertain, and relevant to patients and clinicians. These demands are often in opposition. For example, a RCT would need to enroll 21,000 surgical patients for adequate power to detect a 50% decrease from a baseline surgical site infection (SSI) rate of 2 per 100 procedures. Instead, healthcare epidemiology researchers could choose common outcomes that can be automated, such as hospital-onset bacteremia, instead of rare and/or labor intensive outcomes, such as NHSN-defined CLABSIs. The choice of outcomes in antimicrobial stewardship research is particularly challenging; thus, novel metrics have recently been proposed.Reference Evans, Rubin and Follmann ⁹

Internal validity is essential in determining outcomes, but the best trials achieve both internal and external validity (or generalizability).Reference Friedman, Furberg and DeMets ² Patient-level RCTs often use patients with fewer comorbidities because of either specific inclusion/exclusion criteria or requirement for informed consent (as well as the difficulty of using legally authorized representatives). The outcomes of these patients might not be generalizable to the overall population of patients with the condition or exposure being studied. Healthcare epidemiology studies using cluster randomization, especially randomization that includes all patients in a unit or hospital, overcome some issues of generalizability.

Many interventions in healthcare epidemiology rely on behavioral changes, eg, use of gowns and gloves or hand hygiene. Behavioral change is difficult to study. Thus, the feasibility of making even small changes to clinical behavior and being able to demonstrate its efficacy in a clinical trial is important to consider, particularly if the standard practice is relatively well followed.

The use of a standard protocol and analysis plan are critical for RCTs. Because of the strength of recommendations that can come from RCTs, a high standard is placed on objective documentation of a protocol and statistical analysis plan prior to initiation of the trial (and uploaded to http://clinicaltrials.gov). Although the initial documentation requirements are burdensome, a well-constructed protocol and analysis plan developed in advance and adhered to over the course of the study provide the highest level of evidence and potential impact.

Statistical analysis of RCTs can be complex. A cornerstone of clinical trials is use of the “intention to treat” (ITT) analysis in which participants or clusters are analyzed according to their allocation (ie, intervention or control) regardless of adherence to the intervention. Such an approach is subject to loss of data, misclassification bias, and “lost to follow-up” (attrition bias). Cluster randomization leads to additional statistical complexities in power calculation and analyses such as the need to account for intracluster correlation and lack of independence among individuals within a cluster.Reference Van Ness, Peduzzi and Quagliarello ¹⁰ Nevertheless, ITT maintains study randomization and therefore is essential for causal inference. In contrast, the “per-protocol” (PP) analysis, which involves only participants or clusters with documented completion of the study protocol, may help determine the attributable impact of an intervention, but PP analysis is subject to loss of power and selection bias.

While the design of RCTs reduces the risk of bias and confounding, these limitations still must be considered when designing a study. If allocation of the intervention is not random, then the benefit of an RCT is lost. Likewise, many forms of bias, including inadequate implementation of the intervention or inadequate collection of outcome data, generally bias RCTs toward the null hypothesis. Using a crossover design in a cluster trial limits the problems of clustering. Likewise, for a cluster trial of a “minimal risk” intervention, obtaining a waiver of informed consent is important for feasibility and generalizability.

PUBLISHED RCTS IN HEALTHCARE EPIDEMIOLOGY AND ANTIMICROBIAL STEWARDSHIP

Trials using patient-level randomization generally involve interventions aimed at improving outcomes in the same patient receiving the intervention. When the primary outcome is dependent on an event in an individual participant that can be modified through intervention on that patient, patient-level randomization is appropriate. For example, patient-level trials have evaluated masks to prevent influenza in healthcare workers,Reference Loeb, Dafoe and Mahony ¹¹ chlorhexidine/alcohol surgical scrub for SSI,Reference Darouiche, Wall and Itani ¹² a chlorhexidine sponge for CLABSI,Reference Timsit, Schwebel and Bouadma ¹³ and antimicrobial catheters for catheter-associated urinary tract infection (CAUTI).Reference Pickard, Lam and MacLennan ¹⁴ Others have explored copper surfaces or a bundle of interventions in long-term care.Reference Mody, Krein and Saint ^{15 ,} Reference Salgado, Sepkowitz and John ¹⁶

Trials using cluster-level randomization are more common in HE&AS research. Most interventions in these trials are designed to prevent infection and transmission, and the patient-level assumption that one participant’s outcome is not influenced by another participant’s outcome is violated. Several RCTs have used intensive care unit (ICU) as the unit of randomization. For example, 4 trials explored the use of chlorhexidine bathing in ICUs to decrease the incidence of ICU-related infections;Reference Climo, Yokoe and Warren ^{17 –} Reference Noto, Domenico and Byrne ²⁰ 2 trials evaluated strategies for screening and isolation of patients with multidrug-resistant pathogens;Reference Derde, Cooper and Goossens ^{21 ,} Reference Huskins, Huckabee and O’Grady ²² and 1 trial evaluated the use of universal contact precautions to decrease the transmission of methicillin-resistant Staphylococcus aureus (MRSA) and vancomycin-resistant Enterococcus (VRE).Reference Harris, Pineles and Belton ²³ In 2 trials, individual nursing home was the unit of randomization.Reference Bellini, Petignat and Masserey ^{24 ,} Reference Juthani-Mehta, Van Ness and McGloin ²⁵ In 2 other trials, outpatient clinic was the unit of randomization used to evaluate stewardship interventions.Reference Gerber, Prasad and Fiks ^{26 ,} Reference Meeker, Knight and Friedberg ²⁷ In 2 studies, hospital ward (ie, ICU or and acute care ward) was the unit of randomization used to evaluate the impact of hand hygiene improvementsReference Fuller, Michie and Savage ²⁸ and hydrogen peroxide vapor decontamination.Reference Passaretti, Otter and Reich ²⁹ Finally, in 1 trial, the entire hospital was randomized to evaluate multidimensional infection-prevention interventions in resource-limited settings.Reference Stevenson, Searle and Curry ³⁰

All trials have limitations, and the RCTs cited above are not exceptions. In fact, most large RCTs have included accompanying editorials and letters to the editor outlining these limitations. In addition, most authors document the limitations they observed during the conduct and analysis of the trial being reported. Limitations of HE&AS RCTs can be loosely placed into 3 groups: protocol adherence, outcomes, and generalizability.

Protocol adherence is important because high protocol adherence strengthens the ability to demonstrate a cause–effect relationship between the intervention and the outcome. Protocol adherence, however, can be difficult to measure, depending on the intervention. Several recently published studies failed to monitor protocol adherence.Reference Loeb, Dafoe and Mahony ^{11 ,} Reference Noto, Domenico and Byrne ^{20 ,} Reference Derde, Cooper and Goossens ²¹ The RCT by Milstone et alReference Milstone, Elward and Song ¹⁹ underscores the importance of monitoring adherence. In this trial, chlorhexidine bathing did not lead to a reduction in ICU-related infections in the ITT analysis. Protocol adherence was 64%, however, because 2 participating units required documentation of informed consent and had low participation. Ultimately, the PP analysis demonstrated a significant reduction in ICU-related infections following chlorhexidine bathing.

Outcome limitations are related to inadequate statistical power, ascertainment bias, definitions, plausibility, comparator groups, and multiple interventions performed at the same time. Studies that focus on infrequent outcomes have to enroll many patients to have sufficient power to demonstrate a difference. For example, 2 recently published “negative” studies had limited power related to the low frequency of outcomes.Reference Noto, Domenico and Byrne ^{20 ,} Reference Bellini, Petignat and Masserey ²⁴ Additionally, outcomes must be measured at points that capture the impact of the intervention. For example, Bellini et alReference Bellini, Petignat and Masserey ²⁴ found that a MRSA decolonization regimen was ineffective 12 months after the intervention but conceivably could have been effective earlier.

HE&AS RCTs often take advantage of previously established data collection strategies, such as surveillance data by infection preventionists using NHSN definitions. While this approach simplifies data collection, it potentially introduces “surveillance bias,” a form of ascertainment bias, to the study.Reference Haut and Pronovost ³¹ This point is underscored by the difference between rates of SSI reported to NHSN. For example, rates of colon surgery SSIs are <5% according to NHSN surveillance, but when individual patients are tracked, this rate is >10%.Reference Young, Knepper, Moore, Johnson, Mehler and Price ³² Other RCTs have used “proxy” outcomes or definitions to maximize statistical power or to improve measurability of outcomes. Harris et alReference Harris, Pineles and Belton ²³ used rates of acquisition of MRSA or VRE in their RCTs of universal gloves and gowns in ICUs because they had insufficient power to identify differences in rates of infection. Pickard et alReference Pickard, Lam and MacLennan ¹⁴ used antibiotic therapy as a proxy for CAUTI to improve identification and measurability of outcomes. Finally, some RCTs have used groups of outcomes (ie, any bloodstream infection), but some specific components of these outcome groupings may be more or less relevant (eg, coagulase-negative Staphylococci).Reference Climo, Yokoe and Warren ^{17 ,} Reference Huang, Septimus and Kleinman ¹⁸

RCTs can be limited by differences between the experimental and control groups that the randomization process did not or could not adjust for. For example, Darouiche et alReference Darouiche, Wall and Itani ¹² studied the impact of CHG-alcohol preoperative skin preparation against povidone-iodine for the reduction of SSI. Because alcohol is a highly effective skin antiseptic, it is difficult to determine whether the improved rate of SSI observed with CHG-alcohol was related to CHG, alcohol, or both. Randomization is not perfect; thus, other studies have unintentional differences among study groups despite appropriate randomization. Some recent studies had high baseline rates in clusters randomized to the intervention;Reference Huang, Septimus and Kleinman ^{18 ,} Reference Harris, Pineles and Belton ^{23 ,} Reference Bode, Kluytmans and Wertheim ³³ thus, it is conceivable that at least a portion of the changes observed in these groups was related to regression to the mean.

RCTs in HE&AS interventions have commonly used multiple simultaneous interventions.Reference Mody, Krein and Saint ^{15 ,} Reference Huang, Septimus and Kleinman ^{18 ,} Reference Derde, Cooper and Goossens ^{21 ,} Reference Juthani-Mehta, Van Ness and McGloin ^{25 ,} Reference Gerber, Prasad and Fiks ²⁶ This approach precludes interpretation of the impact of individual components of the intervention. For example, Huang et al. demonstrated that universal CHG bathing and mupirocin decolonization in the ICU leads to lower rates of bloodstream infection.Reference Huang, Septimus and Kleinman ¹⁸ It is unclear, however, whether the impact was primarily due to a single component of the intervention or only to the combination of components.

Finally, generalizability is important to consider for all trials. Limitations related to generalizability are related to patient population and intervention. Some RCTs are performed in settings that are not representative of other practice settings. For example, stewardship interventions that rely on electronic reminders may be difficult to replicate on a different electronic health record.Reference Gerber, Prasad and Fiks ²⁶ Baseline epidemiology of infection is important to consider when evaluating an RCT. For example, studies to reduce SSIs performed in settings where MRSA is not prevalent are difficult to generalize to settings where MRSA is the leading cause of SSI.Reference Bode, Kluytmans and Wertheim ³³ Some interventions may be performed in specific patient populations (eg, postoperative patients); therefore, it is difficult to determine whether the intervention (eg, antimicrobial urinary catheter) will be effective in other patient groups.Reference Pickard, Lam and MacLennan ¹⁴ Finally, some interventions that require significant time or capital outlay are logistically challenging to execute outside of controlled study settings.Reference Salgado, Sepkowitz and John ^{16 ,} Reference Passaretti, Otter and Reich ²⁹

MAJOR TAKE-HOME POINTS

Researchers embarking on an RCT in healthcare epidemiology or antimicrobial stewardship must consider multiple issues (Table 2, outlined in a checklist). All RCTs require careful planning prior to initiation. This pre-study planning must include the development of a study protocol with explicit study questions, randomization scheme, patient/cluster-unit randomization strategies, hypotheses, and outcomes. A biostatistician should compute power calculations and assist with protocol development. CONSORT statements for clinical trials should be reviewed and referenced during protocol development.Reference Campbell, Elbourne and Altman ^{5 ,} Reference Moher, Schulz and Altman ³⁴

TABLE 2

Checklist of Key Considerations When Developing a Randomized Controlled Trial to Evaluate an Intervention in Healthcare Epidemiology or Antimicrobial Stewardship