10.2.1 Lifecycle Regulation and Real-World Evidence at the FDA

The idea that a product’s safety or efficacy profile might change significantly once used widely in the “real world” is far from new. Although controlled experiments help evaluate the safety and efficacy of a product in a target population, they may also mask important risks or exaggerate benefits that become apparent when the product is used over longer periods, in larger populations, and beyond the confines of strict trial protocols. Thus, the FDA has always been somewhat alert to how pre and postmarket experience with a product might differ. Even so, the FDA’s recent shift from pre- to postmarket data gathering and evaluation is both marked and remarkable. New sources of postmarket data are influencing the FDA’s upstream decisions about whether, and on what terms, to approve health products.Footnote ⁴

Of course, postmarket studies that are required by the FDA, or voluntarily undertaken by the sponsor, may still take the form of a randomized clinical trial (RCT). That is not what the FDA and others mean when they refer to real-world evidence (RWE) and real-world data (RWD). The former is essentially any evidence generated outside typical clinical research settings. The latter comes in multiple forms, including “electronic health records (EHRs), claims and billing activities, product and disease registries, patient-generated data including in home-use settings, and data gathered from other sources that can inform on health status, such as mobile devices.”Footnote ⁵ Some researchers are trying to replicate RCT findings using RWD,Footnote ⁶ and within at least some corners of the FDA, including recent FDA Commissioners,Footnote ⁷ the appetite for RWE is growing.Footnote ⁸

One linear account of this change is that Congress and others outside the agency have pushed the FDA toward a “lifecycle” approach to regulation that incorporates RWE. While the process has been mostly gradual, dating back to the HIV/AIDS crisis of the 1980s, the 21st Century Cures Act of 2016 marked a tipping point. The landmark legislation directed the FDA to consider nontraditional study designs and data analysis to streamline drug reviews;Footnote ⁹ apply the “least burdensome means” of approving devices, for instance, by factoring in the likelihood of RWD clarifying safety and effectiveness;Footnote ¹⁰ remove certain medical software from medical devices subject to FDA oversight;Footnote ¹¹ establish an expedited regulatory pathway for “breakthrough” devices;Footnote ¹² and develop guidance to incorporate RWE and patient experience data into its decision making for drugs and devices alike.Footnote ¹³

Another reading of this shift toward lifecycle regulation and RWE suggests that the FDA itself has shaped this regulatory arc. For example, the FDA established the accelerated approval process for drugs and the breakthrough program for devices years before Congressional direction – perhaps to safeguard the agency’s central role in pharmaceutical governance.Footnote ¹⁴

Whatever the motivations, the shift toward lifecycle regulation and RWE remains a work in progress. Postmarket studies regularly take years to completeFootnote ¹⁵ and seldom improve the evidence already gathered.Footnote ¹⁶ The FDA rarely threatens to impose fines or withdraw authorization when postmarket studies are delayed or the evidence does not confirm efficacy.Footnote ¹⁷ Moreover, the FDA’s legal authorities and resources to enforce postmarketing requirements are inadequate and the continuing dominance of the FDA’s reviewing divisions over its postmarket monitoring divisions compromises the agency’s ability to revisit initial decisions.Footnote ¹⁸ Meanwhile, numerous studies show – notwithstanding agency claims to the contrary – that the FDA has been applying a lower regulatory bar for approval of drugs, and the vast majority of medical devices escape formal scrutiny of safety and efficacy.Footnote ¹⁹

In sum, the FDA’s capacity to spur sponsors to generate reliable information about their productsFootnote ²⁰ and to adjust regulatory decisions as the evidence evolves are each in serious question. It is time to consider new mechanisms to counter these shortfalls. The remainder of Section 10.2 details why medical devices, especially digital health products, offer an opportunity for the FDA to pursue this very sort of regulatory experimentation.

10.2.2 Signs of Regulatory Experimentation in Digital Health and Beyond

The FDA’s framework for regulating devices has not changed much since the 1976 Medical Device Amendments, despite radical technological advances.Footnote ²¹ Although a consensus now favors reform,Footnote ²² Congress has done little apart from calling for task force recommendations for how to regulate health IT products,Footnote ²³ and trying to clarify which products fall within FDA jurisdiction.Footnote ²⁴

In the absence of reform, the FDA itself has begun to experiment with new approaches. The agency’s 2017 Digital Health Innovation Action PlanFootnote ²⁵ articulates three key departures from the FDA’s longstanding framework for devices: 1) shifting evidence gathering and evaluation from the premarket to the postmarket phase; 2) scrutinizing firms rather than products, using a new “Software Pre-Certification Program” to evaluate companies offering products; and 3) outsourcing market certification to independent, third-party reviewers, moving away from centralized agency review. While the first departure mirrors the agency’s lifecycle approach to drug regulation, the other two departures are unique, as centralized, product-specific reviews have been the lodestar of FDA regulation for roughly a century.Footnote ²⁶,Footnote ²⁷

Although the details of these new approaches are still in flux, they revolve around a few core ideas. First, shifting evidence gathering to the postmarket setting effectively grants sponsors a kind of conditional or phased authorization, with the expectation that postmarket evidence might confirm the device’s safety and efficacy.Footnote ²⁸ The FDA has assigned the task of gathering such evidence to NEST, the National Evaluation System for health Technology,Footnote ²⁹ a public-private initiative led by the FDA’s Center for Devices and Radiological Health (CDRH).Footnote ³⁰ NEST is charged with collecting RWE from multiple sources, including electronic health records, insurance claims, pharmacy records, device registries, and patient-generated data (PGD).Footnote ³¹ As of 2019, the NEST network includes over 195 hospitals, 3,942 outpatient clinics, and fifteen coordinated registry networks that curate and analyze data.Footnote ³² NEST will organize data “into several standardized common data models (including domains such as demographics, diagnoses, procedures, and laboratory tests).”Footnote ³³ Importantly, while NEST was originally proposed as a way to conduct postmarket surveillance to identify safety issues early,Footnote ³⁴ it has broadened its focus to collecting data throughout the entire product lifecycle, using it not only for postmarket surveillance, but also for premarket review.Footnote ³⁵ For example, the FDA said such evidence could be used to support a sponsor’s petition for device reclassification.Footnote ³⁶ The data could also be used, ideally, to inform insurance coverage and reimbursement decisions, clinical practice, and patient adoption.Footnote ³⁷

In 2018 and 2019, NEST solicited proposals for test cases to evaluate how well such data can be used to answer specific questions.Footnote ³⁸ The latest round includes, for example, a study using insurance claims data to evaluate whether to expand the label for cardiac devices in children with congenital heart disease, and a trial using electronic health records and patient data to evaluate how well the Apple Watch ECG can detect irregular heart rhythms, to inform premarket submissions and postmarket surveillance.Footnote ³⁹ The twenty approved test cases span a range of therapeutic devices (oncology, cardiology, vascular, orthopedic, etc.), a range of risk profiles (from low-risk 510(k) devices to higher-risk PMA devices), a range of data (retrospective and prospective), and a range of proposed uses (premarket, postmarket, and coverage decisions).Footnote ⁴⁰ The test cases will also allow NEST to address concerns over the validity of studies using RWE, with expert committees focusing on the quality of the source data and designing appropriate methodologies for data analysis.Footnote ⁴¹

While the FDA’s shift toward lifecycle regulation has been the subject of growing critique,Footnote ⁴² NEST can add significant scientific rigor to the process of collecting and analyzing RWE to the benefit of “regulatory, clinical, and coverage decision making” not to mention “the health and the quality of life of patients.”Footnote ⁴³ However, there are also reasons to be skeptical that this will occur, underscoring the need for even more radical experimentation, which we describe in Section 10.3.

10.3.1 The “Drug Efficacy Study Implementation” Experiment

Although the 1962 Kefauver-Harris Amendments are widely considered to be foundational, the requirement that drug manufacturers show “substantial evidence” of effectiveness were preconfigured by agency practice. Safety, the sole criterion for market entry from 1938 to 1962, was understood by the FDA to encompass clinical utility or effectiveness beginning in the early 1950s.Footnote ⁴⁴ Administrative innovation pre-staged congressional legislation.

The 1962 amendments likely emboldened the FDA, not only in terms of justifying heightened expectations for evidence of efficacy, but also in terms of using its administrative discretion to fashion solutions to problems perceived in the marketplace. Central among them was the question of what to do about the thousands of “old drugs” that had entered the market between 1938 and 1962, which were not formally evaluated for effectiveness prior to Kefauver-Harris. Congress did not explicitly require the FDA to review these old drugs,Footnote ⁴⁵ but the agency read multiple sections of the legislation as all the mandate they needed.Footnote ⁴⁶ Within a few years “DESI” was born.

DESI would come to evaluate some 3,400 old drugs for over 16,000 therapeutic indications over twenty-plus years.Footnote ⁴⁷ To accomplish that feat, the agency understood that it needed a remarkable new structure. In 1966, under the leadership of Commissioner James Goddard, the FDA contracted the work to the National Academy of Sciences (NAS) and National Research Council (NRC).Footnote ⁴⁸ The FDA not only lacked sufficient personnel for the task, but its personnel lacked the clout that NAS/NRC experts could command if and when difficult decisions had to be made to pull products from the market. The FDA created a centralized Policy Advisory Committee to define DESI’s procedures, which in turn spawned thirty review panels assigned to the therapeutic categories of the day. Each panel was comprised of a chair and approximately six NAS/NRC experts. They worked in confidence, delivering recommendations to the FDA about whether a given drug was “effective,” “probably effective,” “possibly effective,” or “ineffective.” Even though the panels did not conduct new research, each panel reviewed the medical literature for roughly 150 drugs, requiring 10,000 hours of expert scientific labor.Footnote ⁴⁹

DESI drew lawsuits from industry as the FDA followed through on panel recommendations, announcing hundreds of drug withdrawals via the Federal Register.Footnote ⁵⁰ The litigation was less about the involvement of outside NAS/NRC experts, and more to do with the summary-type procedures that the FDA had adopted in the name of efficiency. Notwithstanding firms’ legal challenges, the litigation ultimately failed. The Supreme Court largely validated the agency’s approach in the 1973 “Hynson quartet” of cases involving challenges to NDA withdrawals for preamendment drugs.Footnote ⁵¹ Even without explicit statutory authorization, in Hynson the Supreme Court refers to DESI as a “statutory mandate,”Footnote ⁵² and a Senate Report from 1972 refers to DESI as being “required by the Drug Amendments of 1962.”Footnote ⁵³ Further, the Supreme Court upheld the FDA’s power to use summary procedures, ruling that firms’ expectations of a full administrative hearing to decide the fate of a drug was conditional upon having first produced “substantial evidence” of effectiveness. Where such evidence is lacking, the Court held, a full hearing need not follow.

The implications of DESI are manifold. But the move to engage outside actors in the decision-making process is underexamined. If the FDA’s inability to encourage high-quality evidence production are ultimately reflective of a kind of incumbency – both in terms of who is involved in producing and how it is appraisedFootnote ⁵⁴ – then regulation may take as its inspiration DESI’s disruptive move to bring outside actors into the regulatory fold. In the realm of medical devices, recent FDA initiatives such as NEST show some willingness to do this.

But the success of a DESI 2.0 for devices may depend on coupling 1) third-party evidence generation with 2) third-party reviews of that evidence – two functions which neither the original DESI nor more recent initiatives like NEST have sought to combine. Third-party evidence generation and reviews might significantly strengthen the use of “real-world” signals beyond what the FDA and/or industry is either capable or willing to do, making more meaningful recent odes to total lifecycle regulation and postmarket surveillance.

10.3.2 Repurposing Existing Legal Authorities to Support a “DESI 2.0”

Allowing third parties to both generate evidence and conduct rigorous product reviews is a less radical idea than we might think. Data are now available through many different sources, including massive device registries.Footnote ⁵⁵ And the sheer volume of devices introduced into the market, particularly in digital health, augers in favor of outsourcing some portion of review of safety and efficacy. It is the joining of these two functions and empowering third parties to fulfill them that is crucial.

A threshold question is whether a DESI 2.0 would be legally permissible. Just as the Kefauver-Harris Amendments were interpreted by the FDA as authorizing the original DESI, the current statute is flexible enough to support both device reviews and evidence generation by third parties. First, the statute very broadly requires the FDA to “consider, in consultation with the applicant, the least burdensome appropriate means of evaluating device effectiveness that would have a reasonable likelihood of resulting in approval,” unless “contrary to public health.”Footnote ⁵⁶ Moreover, as with DESI, the statute entrusted initial review and classification of pre-1976 devices to expert panels, and envisioned that the FDA could turn to expert panels to review classification petitions.Footnote ⁵⁷ In both cases, panel decisions are recommendations published and reviewed by the FDA.Footnote ⁵⁸ Likewise, the statute authorizes the FDA to withdraw or suspend PMA approvals, particularly when “new information” is presented.Footnote ⁵⁹ FDA rules make clear that the agency “may seek advice on scientific matters from any appropriate FDA advisory committee” and “may use information other than that submitted by the applicant” in deciding whether to withdraw approval of a PMA.Footnote ⁶⁰

Section 523 of the FDCA provides more specific authority for the kind of dual-function third-party mechanism we imagine. In 1997 Congress amended the statute to codify a five-year pilot program to allow the FDA to accredit third parties to review 510(k)s and make nonbinding recommendations to the agency.Footnote ⁶¹ The FDA initiated the pilot in 1996, before it received statutory authorization.Footnote ⁶² The idea was to provide manufacturers of certain devices “an alternative 510(k) review process that could yield more rapid marketing clearance decisions” and preserve FDA review for higher-risk devices.Footnote ⁶³ The FDA published accreditation criteria in 1998,Footnote ⁶⁴ and the pilot has been renewed by Congress every five years since 2002.Footnote ⁶⁵ Currently, only eight entities are accredited for the renamed “3P Review Program.”Footnote ⁶⁶ Although 510(k) user fees are waived and FDA clearance is 29 percent faster when recommended by an accredited third party, the program remains underutilized.Footnote ⁶⁷

Despite possessing sufficient legal authority to create a DESI 2.0, the FDA might seek more clear statutory authorization from Congress in order to act upon third-party evidence and recommendations. Although the FDA’s authority to adopt summary-type procedures for devices would be supported by the Hynson quartet of Supreme Court decisions, more aggressive reliance on third-party reviews might need clearer statutory support. To wit, after the FDA announced its Digital Health Action Plan and software precertification program, several Senators sent a letter to the FDA questioning the agency’s statutory authority to do so.Footnote ⁶⁸ Indeed, the FDA’s announcement of the Action Plan itself acknowledged that it may lack statutory authority for third-party precertification.Footnote ⁶⁹ However, there is a long history of the FDA relying on panels and advisory committees to make nonbinding recommendations regarding product approvals, classifications, and withdrawals. A similar system, whereby NEST (or some other third party) would be empowered not only to analyze newly collected data but also make recommendations to the agency about appropriate regulatory actions in light of that evidence – ranging from label changes to product withdrawal – would seem to be within FDA authority. Advisories and recommendations are, by their very nature, nonbinding, though their publication would force the FDA to offer compelling justifications for making decisions contrary to the recommendations.

Thus, the stars seem well aligned for a DESI 2.0 for devices. Agency practice presages it. Intense cooperation with third parties to develop new sources of RWE and deploy them for regulatory decisions presages it. The statutory authority remains broad and arcs in that direction. And, perhaps most importantly, the need is clear. The FDA itself remains unable to give adequate attention to the sheer volume and variety of new devices. If vogue ideas like RWE, RWD, and total product lifecycles are to gain real traction, formalizing and inviting third-party participation seems crucial.