With limited healthcare budgets and ever greater complexity of new products and services, health technology assessment (HTA) is an important gateway to diffusion and adoption of new health technologies, and is becoming increasingly important to policy makers in making coverage and reimbursement decisions. In particular, HTAs have become important for the implementation of nucleic acid-based tests (Reference Morrison and Boudreau1;Reference Potter, Avard and Graham2), one of the fastest growing segments of the in vitro diagnostic laboratory business, and key to personalized medicine. To date, breast cancer treatment decisions have largely been determined by clinicopathologic assessment, typically through a single or small set of biomarkers used to direct therapy. The incomplete and inaccurate nature of this information has limited clinicians’ ability to provide personalized therapy. However, over the past decade several multivariate gene expression profile (GEP) tests for breast cancer prognosis (Reference Goncalves and Bose3) have been developed promising a more individualized approach. GEP tests use mRNA expression levels of a small set of genes jointly (multivariately) to predict patient clinical outcome. Two such tests, MammaPrint (the “70 gene prognostic signature”) (Agendia, Amsterdam, The Netherlands) and Oncotype Dx (Genomic Health, Redwood City, CA), have been commercially available for several years. The increased accuracy of GEP tests to predict cancer recurrence provides better estimators of whether patients will benefit from specific therapies. The avoidance of unnecessary chemotherapy results in decreased toxicity for patients, and consequent cost-savings associated with reduction in erroneous chemotherapy use (Reference Goncalves and Bose3). For these reasons, prognostic-based gene expression tests are of great interest to insurers and healthcare systems (4), as well as clinicians and patients.
HTA has been described as a “multidisciplinary field of policy analysis study(ing) the medical, social, ethical, and economic implications of development, diffusion, and use of health technology” (5). Numerous HTA organizations and experts have called for explicit integration of ethical, legal, and social issues (ELSI) in HTAs (Reference Potter, Avard and Graham2;5, European Network for HTA (EUnetHTA), see www.eunethta.eu and www.corehta.info). However, research shows most assessments concentrate on clinicotechnical and cost-effectiveness issues (Reference DeJean, Giacomini, Schwartz and Miller6). Furthermore, some have suggested that the divide between ELSI and clinicotechnical issues is arbitrary; in practice these issues are deeply integrated. Notably, key frameworks for genetic test evaluation reflect these assertions (Reference Morrison and Boudreau1;Reference Potter, Avard and Graham2). For example, ACCE and EGAPP processes address ELSI as part of the fundamental clinico-technical measures of analytical and clinical validity, and clinical utility (Reference Teutsch, Bradley and Palomaki7;8). Likewise, the UK Genetic Testing Network underlines context, which includes organizational and socio-ethical aspects, as critical in defining test purpose and feasibility, and thus clinical utility (Reference Burke and Zimmern9).
Often, ELSI can arise in the clinical application of a new technology or during clinical translation. Generally ELSI surrounding genetic testing stem from the sensitive personal and familial nature of hereditary information. However, Omics tests, including GEP, can raise the stakes because they are used to direct personalized treatment (Reference Ali-Khan, Daar, Shuman, Ray and Scherer10). While discrimination and privacy concerns may continue to be relevant, GEP tests may raise novel issues. For example, GEP tests assess the expression of multiple genes which are then algorithmically processed to yield a single patient-specific result. The derivation of this resulting risk score is nontransparent (11). Thus, a clinician using a GEP test is not able to independently derive or verify the outcome, a fact that challenges current clinical paradigms (Reference Mokhtar, Murad, Mian, Jamal, Lopez-Camarillo and Arechaga-Ocampo12), and can promote clinician unease (Reference Zhang13). Equally, this complexity may challenge clinicians’ ability to explain tests and hamper patients’ understanding of the results, risks and benefits of treatment options. Despite the recognized importance of ELSI in the context of Omics research and clinical application, there is limited understanding of the extent to which they are addressed in “personalized medicine” Omic-based HTAs. Our goal is to explore current practice in the integration of ELSI regarding GEPs for breast cancer prognosis, an early example of a personalized medicine technology whose clinical use is expanding. The specific objectives of this study are to: (i) identify the ELSI occurring in HTAs of GEP tests for breast cancer prognosis, (ii) assess whether these ELSI are addressed implicitly or explicitly (Reference DeJean, Giacomini, Schwartz and Miller6), and (iii) examine the methodology adopted to assess ELSI.
METHODS
Search Strategy
To facilitate a structured and transparent approach to identifying HTAs we developed a systematic search protocol using relevant guidance (14), and assisted by a professional medical librarian (see Acknowledgements). Keywords, MeSH terms and synonyms for use in search strings were identified using the National Library of Medicine's MeSH browser (see, http://www.nlm.nih.gov/mesh/MBrowser.html), and by “snowballing” vocabulary from several articles in the breast cancer and gene expression profiling field (see Supplementary Information). The resulting search terms fell into four conceptual categories and included variations of “gene expression profiling,” “prognosis,” “breast cancer,” and “health technology assessment.” Electronic searches of forty-seven resources were carried out including: the Cochrane Library, EMBASE, CINAHL, HTA agency Web sites and databases, grey literature and general search engines, limited to publication January 1, 2004 to August 31, 2012. This start date was selected because the first GEP test for breast cancer became commercially available in 2004 (see Supplementary Table 1 for search strings and databases). Handsearching of several retrieved HTAs was also completed (Reference Smarrt15–19).
Selection of Publications
We defined HTAs as documents providing a technology assessment of GEP tests for breast cancer prognosis, including horizon scans and full assessments, published by international, national and regional HTA-producing entities or authors affiliated with these. Reports in languages other than English, those produced by private payers only available for a fee, and policy documents based on HTAs were not included. Documents describing themselves as solely economic or cost-effectiveness assessments (CEAs) were also excluded from this analysis, as CEA are usually considered to be a component of, but not themselves HTA. While social value considerations may be integrated in CEAs, this more focussed analysis was not the aim of the current study. Initial screening of search strategy hits was based on titles and abstracts. If there was no abstract or the information in the abstract was insufficient for decision making, the full-text document was obtained for closer examination. Sixty-nine documents from a total 1,358 search hits were identified for full-text review, and downloaded or requested from authors. After doubles were removed (Reference Droste, Dintsios and Gerber35), the full texts of the thirty-three remaining documents were vetted for final inclusion in our dataset. In total in the two examination phases, two non-English language, thirteen CEA and six pay-for-view documents were excluded. An evidence report and overview, and a provisional document for one HTA still in preparation during the study period were identified (4). After examination, only the latter (22), being the penultimate and thus most likely to include socio-ethical content was retained. Likewise, the content of one HTA (Reference Marchionni, Wilson and Marinopoulos25) was subsequently published as a systematic review. Only the HTA was retained (see Supplementary Information). For Startpage (>41,000 hits), only the first 1% of records were examined. Two authors carried out this examination independently (S.E.A. and L.B.). Review of included/excluded document abstracts and, if needed, the full text was carried out by a third author to verify the final study dataset. Any discrepancies were discussed by at least two of the authors to reach consensus. This process yielded seventeen HTAs for analysis (Figure 1).
Shown is the study work flow showing the number of health technology assessments (HTAs) identified at each step in the systematic collection process.
Data Analysis
Descriptive features from the abstracts and full text were coded including the country and agency of origin, publication year, technologies or tests assessed, the stated scope of the HTA (expressed either as a formal aim/goals/scope statement or as section headings), and funding information (if not stated, the issuing organization web site was examined for these details).
In this study we focussed on material in the HTAs assessing MammaPrint and Oncoytpe DX, as they were the only multivariate tests for breast cancer prognosis on the market during the study period. Three aspects were explored. First, we identified ELSI content guided by Hofmann's framework, a checklist designed to aid in integrating moral issues in HTAs (Reference DeJean, Giacomini, Schwartz and Miller6;Reference Hofmann20) (see Supplementary Table 2). Hofmann's thirty-three questions address broad considerations, including implications of the technology, its purpose and implementation, and issues related to stakeholders and the HTA methodology. Although morals and ethics are not equivalent concepts, this checklist used in concert with prior knowledge of the literature concerning ELSI and Omic technologies, provides an amenable conceptual framework for analysis (Reference DeJean, Giacomini, Schwartz and Miller6). In this text, we use the terms ELSI and the narrower socio-ethical issues inter-changeably. HTAs were examined for text relevant to each of the thirty-three question areas. We also looked for other material relevant to ELSI and to HTA for GEP tests, for example whether conflict of interest and ethical approval information was listed for studies included in HTAs. Given the extensive contemporary debate about evidentiary requirements for molecular testing, we also examined documents for these issues as well as other pertinent or recurring themes. Relevant text was extracted to Microsoft Excel (Mac version 14.4.6 software). Thematic qualitative content analysis and the constant comparative technique was then used to progressively code ELSI, and group them into themes (Reference Braun and Clarke21). Second, we examined whether the extracted ELSI were explicitly framed (i.e., explicitly labelled as “ethical,” for example appearing in a section entitled “socio-ethical analysis”), or whether they were embedded within the HTA without acknowledgment of their socio-ethical dimensions (i.e., implicit) (Reference DeJean, Giacomini, Schwartz and Miller6). Third, for these explicit HTAs we examined the methods given by authors for identifying ELSIs. These data were collected by one author (SEA), verified by a second author (D.A.), and any disagreements were discussed to consensus.
RESULTS
Characteristics of HTAs and Examination of ELSI Content
Our systematic search identified seventeen HTAs (Table 1 and 2). More than half were produced by U.S. organizations, three were produced by Australian/New Zealand and British groups respectively, and one each were produced by Malaysian and Canadian agencies. All of them assessed either the MammaPrint or Oncotype DX tests, with some variously reviewing other GEP or breast cancer prognostic tests in addition. All of the HTAs that described their scope or aims included assessment of analytical and clinical validity, and clinical utility, or effectiveness or efficacy in improving clinical outcomes. Only three (18%) of the HTAs stated that examination of socio-ethical issues was one of their aims and only these documents explicitly addressed such considerations (Table 1) (hereafter these are referred to as “explicit HTAs”). However, our analysis showed that all of the HTAs in our dataset included material with implicit socio-ethical implications (Table 1 and 2). Furthermore, one HTA presented “contextual issues” (22). While the word “ethical” was not used in this section of the HTA, our analysis indicated that it included implicit socio-ethical material (Table 2).
ELSI Themes in HTAs with Explicit ELSI Sections
Note. Shown are descriptive features of the three study HTAs which presented ELSI ‘explictly,’ .ie., in sections entitled ‘ethical issues,’ and a summary of the ELSI mentioned therein.
ELSI Themes Implictly Present in Study HTAs
Note. Shown are descriptive features of the fourteen study HTAs in which ELSI occurred implictly, i.e., they were not framed as having ethical implications by authors, and a summary of the ELSI mentioned therein.
ELSI Raised by Gene Expression Profiling Tests for Breast Cancer Prognosis
We then examined the ELSI constituting the explicit and implicit content in the study HTAs. Our analysis identified eight key themes and two sub-themes: (i) clinician-patient communication and informed consent; (ii) discrimination; (iii) privacy; (iv) psychosocial issues; (v) patient-centered issues including potential benefits and harms; sub-themes: deficiencies of evidence; patient risk tolerance; (vi) conflict of interest; (vii) equity and access issues; and (viii) healthcare delivery, distribution and re-assessment issues. Some ELSI appeared only within explicit ELSI sections, some occurred both explicitly and implicitly, and a few were only handled implicitly in HTAs (Tables 1 and 2).
ELSI Framed Explicitly only
There was some, but not extensive overlap in the ELSI themes and issues that were explicitly framed in the three explicit HTAs. Many of these issues have been widely discussed in the context of genetic and Omic testing, and thus could be considered “classical” ELSI. In particular these included informed consent, discrimination, privacy, and psychosocial issues, and concerns stemming from lack of patient or clinician genetic literacy (under patient-centered issues and clinician-patient communication issues themes, respectively).
ELSI Occurring Both Explicitly and Implicitly
Some ELSI were presented both explicitly and implicitly, including patient-centered issues, conflict of interest, equity/access and healthcare delivery, distribution and re-assessment issues (Tables 1 and 2). Some of these ELSI were more novel and specific to the personalized medicine model, patient stratification based on individualized parameters, or the nature of GEP testing. In one HTA these were presented in an explicit “socio-ethical” section (23), and in the other implicitly in a “contextual” section (22). These ELSI include the concern that patient groups who do not “qualify” for standard treatment based on the results of GEP tests may experience themselves as “orphan-disease” populations (equity/access-related issues) (23); uncertainty about the best way to present GEP risk scores to optimize patient decision-making and benefit (patient–clinician communication issues) (23); the need to better understand patients’ individual risk tolerances and intervention preferences (patient-centered issues) (22;23); and the concern that economic incentives may impinge on proper use of diagnostic/prognostic test therapeutic combinations (healthcare delivery, distribution, and re-assessment issues) (23).
Assessing the potential risks and benefits of a technology is the central focus of HTA. While the harms and benefits of GEP tests have strong socio-ethical dimensions, our analysis showed these were almost always left implicit in study HTAs. For example, a key ELSI raised in almost every HTA was whether test results can be relied upon to withhold chemotherapy in patients who would have otherwise received it based on current standard of care. Thus, all the study HTAs underlined a significant risk of cancer recurrence or death in patients who may forgo chemotherapy based on a GEP test result, but would have actually benefited from treatment (i.e., false negative results) (18;22;24–Reference Ishibe, Schully, Freedman and Ramsey27). This issue was left implicit in all but one HTA (19). Likewise, the potential benefits of testing were largely handled implicitly. Only the personalization of cancer therapy was presented within an explicit ELSI section, and only in one HTA (19) (under patient-centered issues, potential benefits and harms; Tables 1 and 2).
ELSI Occurring only Implicitly
Some ELSI-related material occurred only in an implicit format in HTAs. These included “deficiencies of evidence” (a sub-theme of patient-centered issues), an issue that was present in all the study HTAs; uncertainty about the capacity for GEP test provision to be scaled-up while maintaining analytical validity (23); the need for ongoing comparative effectiveness studies (both in healthcare delivery, distribution and re-assessment issues) (22;26); and potential conflicts of interest (Reference Smarrt15;23) (Table 2).
The largest amount of implicit socio-ethical material in study HTAs concerned deficiencies in the evidence for analytical validity, clinical validity, and especially the clinical utility of the tests under review. This material occurred within the clinico-technical assessment presented in all of the HTAs. Key concerns discussed included: the quality of the archived tumour samples used for gene signature development; statistical “over-fitting” of data in early studies; methodological weaknesses in clinical study designs; the retrospective nature of development/validation studies; and the small number or heterogeneity of patient samples used. Some HTAs also raised concerns more specific to analytical validity (for example, the fact there is no “gold standard” clinical technology for direct comparison of GEP test performance, thus obviating the assessment of analytic false positive/negative rates [Reference Smarrt15;22;Reference Marchionni, Wilson and Marinopoulos25;Reference Ishibe, Schully, Freedman and Ramsey27]), clinical validity (for example, some HTAs questioned the nature of the mechanism by which gene expression values relate to clinical outcome, and/or the methods by which predictive gene signatures are selected [26;Reference Tice28]), and clinical utility (for example, all study HTAs noted a paucity of data on how the results of GEP tests are used in clinical decision-making, and whether test-guided practice improves health outcomes beyond standard clinical practice). No HTA noted or discussed the current uncertainties around evidence requirements for regulatory approval and how these may have affected the characteristics of the studies assessed in HTAs.
Methods to Address ELSI Issues in HTAs
The methodology used to integrate ELSI into HTAs influences the resulting product (Reference Potter, Avard and Graham2), and will impact the characteristics and depth of the presented material. Thus to gain greater perspective on the scope and limitations of the ELSI material in our dataset, we examined the three explicit HTAs for descriptions of the methodology used (as implicit HTAs did not include dedicated ELSI sections, we did not examine these). Despite a variety of relevant methods that exist, those used to identify ELSI were described in only one of the three explicit HTAs (23), being secondary literature review. However, the method was not described in sufficient detail to be reproducible. The referencing/format of one of the other two HTAs suggested that secondary literature review was also the basis for that ELSI section, although this was not stated (Reference Mundy, Braunack-Meyer and Hiller29). In the third HTA, the ELSI section was not referenced, leaving its genesis unclear (19). None of these HTAs described the expertise of those who compiled the socio-ethical material, or details of the inclusion/exclusion criteria used to select the identified references. Finally, while all three HTAs briefly summarized ELSI pertaining to the technologies under assessment, they did not provide ethical analyses or strategies to address the issues raised. Neither did they specifically note whether the issues mentioned were theoretical, or came to light through empirical work.
DISCUSSION
The aim of this study was to catalogue ELSI content in HTAs of GEP tests for breast cancer prognosis and examine how this material is integrated in HTAs. We note a caveat in that our analysis was limited to English language publications with more than half produced in the United States. However, Europe has a strong tradition of socio-ethical consideration in HTAs (Reference Burke and Zimmern9). Thus, inclusion of documents in languages other than English may have produced different findings.
Our analysis indicates that study HTAs contained a large amount of ELSI-related material. A variety of issues featured, ranging from “classical” ELSI to more novel personalized medicine and prognostic GEP test-specific issues. “Classical” ELSI always occurred in explicit ELSI sections. However, much of the material relating to risks and benefits of prognostic GEP tests occurred in clinico-technical sections of HTAs, with its ELSI dimensions left implicit. Less than a fifth of study HTAs included dedicated ELSI sections, indicating that explicit consideration of ELSI is still infrequently carried out. As such, important contextual information that may support integration in healthcare systems, optimize patient benefit, and pre-empt harms was left ambiguous, or not addressed. Experts have noted that bioethical analysis involves “advancing and examining arguments about what ought, morally, to be done and not done—about what is (actually, rather than merely thought to be) right and wrong” (Reference DeJean, Giacomini, Schwartz and Miller6;Reference Benatar30). However, none of the few explicit ELSI sections in study HTAs included such in-depth reflection, nor analysis or recommendations. Rather, HTAs seemed to skim over the issues without providing a roadmap for what should actually be considered by decision makers, or how. The methods used to generate or gather knowledge will influence the nature of results (Reference Potter, Avard and Graham2), while the detailed reporting of methods supports understanding of the scope, limitations, and importance of findings. Strikingly, our analysis showed that methodology for inclusion of ELSI was only described in one HTA. Inclusion of a thorough description of methods, including the training of those undertaking this work, would promote the transparency and comprehensiveness of ELSI material in HTAs. With accelerating development of new prognostic and diagnostic tests, increased attention is focusing on evaluation of these devices (Reference Newland31;Reference Hogarth32). At the same time, regulatory channels remain problematic, and HTA are playing an increasingly important role in weighing the risk and benefits of new tests (4;Reference Newland31;Reference Hogarth32). As such, these documents should be as informative and specific as possible to maximize their utility for decision-making. Our analysis found ELSI encompassing a spectrum of issues important to the overall benefit of prognostic GEP tests, many of which were left implicit. Thus, our results reiterate calls for explicit integration of ELSI into HTA (2;5, European Network for HTA (EUnetHTA), see www.eunethta.eu and www.corehta.info). Overt presentation of ELSI may have multiple benefits—most practically, drawing policy-makers’ attention and increasing the likelihood that ELSI would be considered in decision making (Reference DeJean, Giacomini, Schwartz and Miller6;Reference Hofmann20). To maximize relevance and utility, evaluation of ELSI and related contextual questions should be integrated with clinico-technical considerations in the assessment process, rather than as an ad hoc activity (Reference Teutsch, Bradley and Palomaki7;Reference Burke and Zimmern9;Reference Hofmann20). Our finding of ELSI-related material throughout clinico-techical sections of HTAs reiterates the pragmatism of this approach. Furthermore, ELSI should be placed in context, highlighting their applicability. Thus, the derivation of conclusions or recommendations could augment accessibility for decision makers. Finally, primary ethical analysis or the commissioning of primary empirical studies could increase relevancy of ELSI within the context of each jurisdiction, incorporate stakeholder perspectives, and perhaps uncover novel issues. This type of comprehensive approach may also better link developers, assessors, and end-users, thus streamlining the translational pathway. Semantics may also play a role in mediating the accessibility of ELSI. Notably, a section entitled “contextual issues” in one HTA included much socio-ethical material (22). It is possible that scientists or policy-makers may instinctively consider ELSI less important or less definitive than clinico-technical material. If so, framing issues as “contextual” rather than “ethical” could affect the uptake of this information. Gauging the impact of semantics would be an interesting empirical study.
Much of the socio-ethical material uncovered in this study related to the fact that prognostic GEP tests for breast cancer are commercially available, yet the scientific evidence is still evolving. However, absent from study HTAs was mention of current regulatory debates around Omic tests, issues that are closely tied to technology uptake. Including this information may have assisted in placing evidentiary deficiencies in context, and spurred policy-makers to discussion. Likewise, largely absent from HTAs were clinician and patient perspectives on GEP tests. Exploration of how prognostic GEP tests may challenge current clinical workflows and clinician roles (Reference Mokhtar, Murad, Mian, Jamal, Lopez-Camarillo and Arechaga-Ocampo12;Reference Kohli-Laven, Bourret, Keating and Cambrosio33), how clinicians understand and communicate GEP-derived risk scores (Reference Zhang13), and how they may feel about denying chemotherapy, effectively offering no treatment to a patient if a test indicates this, are all highly relevant to adoption of these technologies, and critical determinants of patient benefit. Likewise, inclusion of patient perspectives is important for relevancy, and for facilitating patient benefit and uptake. That clinician and patient viewpoints were not represented in study HTAs may reflect the use of secondary literature review as a methodology, and the fact that GEP tests are relatively new with few ethical analyses yet published. However, the use of a framework such as Hofmann's that guided our analysis (Reference Hofmann20), would have drawn attention to this gap. Recently, further strategies for integrating ELSI into HTA have been published, including stakeholder consultation (for example Reference Droste, Dintsios and Gerber35;Reference Gagnon, Desmartis and Gagnon36). Thus, study HTAs presented only part of the pertinent information, compelling readers to “read between the lines” or synthesize from external sources.
In conclusion, despite a wide variety of important ELSI occurring in study HTAs, these were rarely explicitly addressed. If the goal of HTA is to support the well-being of patients and the public, then HTA producers have an ethical responsibility to produce material that supports optimal decision making by policy makers. Explicit treatment of ELSI would increase their accessibility to decision makers, and may augment HTA efficiency maximizing their utility. This is particularly important where complex Personalized Medicine applications are rapidly expanding choices for patients, clinicians and healthcare systems.
SUPPLEMENTARY MATERIAL
Supplementary Tables 1 and 2 http://dx.doi.org/10.1017/S0266462315000082
Supplementary Information http://dx.doi.org/10.1017/S0266462315000082
CONFLICTS OF INTEREST
Dr. Ali-Khan reports grants from Genome Québec through the Concours GQ en santé humaine 2010 competition: Next generation predictive signatures for breast cancer, during the conduct of the study. None of the other authors report conflicts of interest.
