Biomarkers are key to personalizing medical care and accelerating the development of new drugs and other medical products. Sophisticated technologies permit assessment of biomarkers on unprecedented scales in terms of numbers and types of measurements. These measurements may capture biology at the DNA, RNA, or protein levels; imaging or digital monitoring may allow assessment of biomarkers reflecting physiologic structures, processes or functions. Coupling these rich data with advanced analysis methods, there would seem to be endless opportunities for biomedical discoveries leading to new therapeutic approaches. Yet, validation of a single biomarker or constellation of biomarkers (i.e., a “signature”) as suitable for a particular purpose is often a painstaking process fraught with challenges. This commentary reflects on the complexities of biomarker validation, including the multitude of issues and challenges highlighted by authors of the articles in this special issue of the JLME.

“Biomarker validation” is a phrase often spoken but seldom adequately defined. The word “validation” seems to project some notion of credibility or credentialing, yet it is a vacuous term in the absence of context. Not until specific role and context for a biomarker have been proposed can a sensible discussion of validation occur. In biomedical research, a helpful first cut is the distinction between use of a biomarker as a medical product development tool and use of a biomarker (or more precisely, a clinical test based on it) to guide clinical care decisions. Articles in this issue touch on both of these categories. Discussion here aims to clarify concepts and emphasize the criticalness of biomarker role and context when assessing validation status of a biomarker.

As defined by the FDA-NIH BEST glossary,¹ medical product development tools are “methods, materials, or measurements used to assess the effectiveness, safety, or performance of a medical product.” Examples of biomarker-based tools include those used for enrichment for clinical trial eligibility, for early detection of drug-related safety signals, and as surrogate endpoints. Several articles in this issue commented specifically on biomarkers as surrogate endpoints, which are endpoints “used in clinical trials as a substitute for a direct measure of how a patient feels, functions, or survives.”² The U.S. Food and Drug Administration uses the term “qualification” to describe “a conclusion, based on a formal regulatory process, that within the stated context of use, a medical product development tool can be relied upon to have a specific interpretation and application in medical product development and regulatory review.”³ Qualification has a similar flavor as the notion of validation but with specific regulatory requirements and implications. FDA has shared its current views on evidence needed for biomarker qualification in a recent publication.^{Reference Leptak, Menetski, Wagner, Aubrecht, Brady, Brumfield, Chin, Hoffmann, Kelloff, Lavezzari, Ranganathan, Sauer, Sistare, Zabka and Wholley4}

FDA's thinking on evidentiary criteria specifically for surrogate endpoints continues to evolve and undergo clarification. Discussions have occurred in public forums,⁵ resources are in development,⁶ and recently the FDA published a table of surrogate endpoints that were the basis of drug approval or licen-sure.⁷ Important considerations mentioned in text accompanying the FDA surrogate endpoints table are: “The acceptability of these surrogate endpoints for use in a particular drug or biologic development program will be determined on a case-by-case basis. It is context dependent, relying in part on the disease, studied patient population, therapeutic mechanism of action, and availability of current treatments. A particular surrogate endpoint that may be appropriate for use in a particular drug or biologic clinical development program, should not be assumed to be appropriate for use in a different program that is in a different clinical setting.” Hey and colleagues note in this special issue that it would have been helpful if the FDA's surrogates table had explicitly specified the definitive clinical endpoint for which each surrogate was considered an acceptable substitute.^{Reference Hey, Feldman, Jung, D'Andrea and Kesselheim8} This is a critical component of context of use for a surrogate endpoint. Further elaboration on the surrogates table would be a welcome addition. Lack of clear description of context of use and proper alignment of evidence to that use risks misapplication of biomarkers, potentially leading medical product development efforts down wrong paths or harming patients.^{Reference Hey, D'Andrea, Jung, Tessema, Lou, Gyawali and Kesselheim9}

Biomarkers underpin a large majority of clinical tests used to guide clinical management decisions for patients. Processes to validate a biomarker-based test for clinical use and to qualify a biomarker as a medical product development tool accepted by regulatory authorities have several aspects in common, but there are also important differences. Three main criteria are essential for validation of a clinical biomarker test: analytic validity, clinical validity, and clinical utility.^{Reference Teutsch, Bradley, Palomaki, Haddow, Piper, Calonge, Dotson, Douglas and Berg10} Analytic validity refers to establishing that the performance characteristics of the locked down test (i.e., completely specified) are acceptable in terms of sensitivity, specificity, accuracy, linear range, precision, reproducibility, robustness to pre-analytic factors, as applicable. Clinical validity refers to demonstrating that the test result is associated with a clinical outcome or characteristic of interest. Clinical (medical) utility refers to establishing that use of the test, when applied according to the intended use, results in a favorable benefit-to-risk balance for the patient, i.e., the patient has a better overall outcome when the biomarker test is used to guide clinical management than when it is not.

Unlike drugs, which must undergo regulatory review to establish safety and effectiveness for clinical use, regulatory oversight of many biomarker tests is minimal. When a biomarker test is used in clinical care decisions, CLIA regulations¹¹ require that the laboratory offering the test is certified to meet certain operating standards. CLIA certification implies that the laboratory has demonstrated that it operates under good laboratory practices, with appropriate quality assurance procedures and adequately trained personnel. While this should help to ensure that biomarker tests offered by these laboratories meet reasonable standards for analytic validity, CLIA certification does not focus on evidence for clinical validity or utility. Nor have the vast majority of clinical biomarker tests ever been reviewed by the US FDA, as many are offered under the setting of laboratory-developed tests where FDA often exercises enforcement discretion in its oversight; however, FDA does have the authority to halt marketing of clinical tests if problems with them come to its attention. An important class of biomarker tests that are regularly reviewed by the FDA are companion diagnostics used to select patients who are most likely to benefit from a particular drug or class of drugs. Although, regulatory oversight of these companion diagnostics is in some sense a spillover from regulatory oversight requirements for drugs. Consequently, the biomedical research community and related professional societies and guidelines bodies bear a heavy responsibility for evaluating clinical validity and utility of many biomarker tests in use. This situation has led to fragmented and sometimes inconsistent processes and standards for validation of biomarker tests.

Many biomarkers (or tests based on them) fall in the intersection of those with potential value as drug development tools and those with potential to underpin a biomarker test having clinical utility. Comparison of evidentiary requirements for these two situations helps to illustrate the important role of context for biomarker validation or qualification. Prognostic and predictive biomarkers often fall in this intersection. Prognostic biomarkers are those “used to identify likelihood of a clinical event, disease recurrence or progression in patients who have the disease or medical condition of interest.”¹² Predictive biomarkers are those “used to identify individuals who are more likely than similar individuals without the biomarker to experience a favorable or unfavorable effect from exposure to a medical product or an environmental agent.”¹³ Biomarkers serving in these roles may be used in a drug development program to enrich the population of patients eligible for a clinical trial of an experimental therapy to assess whether it improves outcomes relative to standard therapy for the disease of interest. A prognostic biomarker could be used to enrich the trial population for those patients at greatest risk of unfavorable outcomes or events. This strategy not only focuses on those patients who might most need new therapies, but it can lead also to a clinical trial that is smaller or of shorter duration because the number of observed events drives the statistical power of a clinical trial with a time-to-event endpoint. The same prognostic biomarker might be used somewhat differently when incorporated into a clinical test. For example, in the adjuvant setting for some early stage cancers, there is great interest in identifying patients who have such favorable clinical outcome following surgery alone, that they could forgo adjuvant systemic therapy. The analytical performance of the biomarker test, the exact way in which the biomarker is measured and test results are interpreted, including any cut-points applied to the biomarker value for clinical decision-making, and other clinical or pathological characteristics of the patients must all be considered to ensure an appropriate benefit-to-risk balance for patients. Analogous considerations apply for predictive biomarkers, which when used for clinical trial enrichment, may increase the odds that enrolled patients have potential to benefit from the experimental therapy. If the clinical trial is positive, it is possible that further refinements would be needed before a test based on the biomarker would be deemed validated for use as a companion diagnostic in routine clinical practice. These scenarios illustrate the need for a clear understanding of context of use, availability of sufficient data of the right types, and use of analysis approaches that appropriately evaluate the risks and benefits inherent in the proposed use of the biomarker.

Biomarker study publications typically serve as the primary vehicle to advertise the existence of new biomarker data and discoveries, yet they often fall short of providing the type of information needed for determination of whether data and results generated by a study support validation. Key information, such as the characteristics of the study population, definitions of study endpoints, details of biomarker assay methods and statistical analyses, and clear distinction between pre-specified versus post hoc (data-driven) hypotheses are often not fully reported in published articles, yet this information is essential to assess clinical context and strength of evidence. Better reporting of biomarker studies,^{Reference McShane, Altman, Sauerbrei, Taube, Gion, Clark, Altman, McShane, Sauerbrei and Taube14} and health research studies in general,¹⁵ would make study results more interpretable and facilitate identification of studies that are pertinent to biomarker validation efforts.

Advances in technologies to measure and collect biomarker data have led to generation of large volumes of increasingly complex data that may provide key biological insights, yet these data present challenges for capture, storage, and retrieval for use in biomarker validation efforts and other studies. Increased openness to (and requirements for) sharing of research data, more resources to store and maintain biomedical data and make it accessible in an ethically appropriate manner (including data from routine clinical care records), and development of new analytical tools capable of handling large and complex data sets, all represent steps in the right direction. Li and Sim^{Reference Li and Sim16} make a strong case for how platforms for sharing data from clinical trials, which are generally the most comprehensive and high quality data, can advance the study of biomarkers. Research funders can play an important role in making clinical trial data available, as evidenced by the recently launched U.S. National Cancer Institute NCTN/NCORP Data Archive.¹⁷ The Data Archive currently has individual patient-level data available for request from dozens of phase III cancer clinical trials, including some in the process of being linked with biomarker and imaging data. In contrast, biomarker investigations conducted on “convenience” specimen sets, often lacking any semblance of study design or having basis in a defined clinical cohort, tend to produce data and results of low value for translational research and biomarker validation efforts.^{Reference Simon, Paik and Hayes18} Unfortunately, many of the earliest studies from which large-scale biomarker data such as omics data were made publicly available utilized convenience specimen sets, and studies of convenience remain quite common in biomarker research.

Simply making more data and analysis tools available may not be sufficient to move many biomarkers toward validation. Statistically significant associations between biomarkers and clinical outcomes together with suggestion of an intriguing clinical or biological hypothesis may be sufficient to support a publication; however, a huge chasm exists between published biomarkers and validated (or qualified) biomarkers. Successful validation or qualification of a biomarker requires a carefully crafted context of use and access to data for the biomarker measured in the right way, at the right time, on the right specimens from the right patients, and accompanied by key clinical and pathologic variables, treatment information, and clinical outcome data. Mavergames and colleagues^{Reference Mavergames, Beecher, Becker, Last and Ali19} describe efforts of the Cochrane Collaboration to curate and synthesize evidence which may support biomarker validation, explaining how Cochrane's Linked Data Project could help advance understanding of surrogate endpoints. Shrager and colleagues^{Reference Shrager, Shapiro and Hoos20} argue that “big data” and “artificial intelligence” methods applied naively are not likely to lead to meaningful medical advances, in part due to the complexity of some diseases like cancer and the “curse of dimensionality.” Instead, they favor informed data reduction by capture of treatment rationales that incorporate expert knowledge and clinical context. These discussions emphasize that biomarker validation is a complex evidence evaluation process that requires more than volumes of data and advanced data analysis tools.

Fostering a biomedical research environment that is more conducive to biomarker validation will require concerted effort. Resources will be needed to develop, manage, and make available in useful form, large collections of rich data and analytic tools. Collaborations among experts representing multiple clinical and scientific disciplines, in consultation with regulatory authorities, will be essential to support and guide biomarker validation efforts to achieve more efficient medical product development and improved clinical care that leads to better outcomes for patients.