Background
Many countries undertake health technology assessment (HTA) to guide decisions about the public reimbursement of new health technologies (e.g., pharmaceuticals, medical services, etc.). Historically, Australia has been at the forefront in integrating government decision making with the assessment of health technologies using HTA (Reference Hailey1). The Medicare Benefits Schedule (MBS) and the Pharmaceutical Benefits Scheme (PBS) are the primary means by which the Australian Government subsidizes new technologies. The process involves submission of HTA applications to independent decision-making committees commonly by industry sponsors or by HTA organizations producing reports on behalf of the government, medical bodies, or industry (applications may provide a letter of support by patient groups). Although national advisory committees can consider submissions from patient groups, data requirements and “cost recovery” (i.e., the government charges) may act as barriers to such submissions. Submissions are then reviewed by HTA evaluation groups commissioned by the Australian Government. The initial submission and the evaluations are considered by two national decision-making bodies, that is, the Pharmaceutical Benefits Advisory Committee (PBAC) and the Medical Services Advisory Committee (MSAC). PBAC and MSAC recommendations regarding medicines and medical services, respectively, are provided to the Minster for Health to inform government decisions regarding the pricing and listing of these technologies on the PBS and MBS.
In recent years, public funding of health technologies has been a major driver of health spending in Australia. In 2017–18, for example, more than AUD45 billion was spent on the MBS and PBS, accounting for 46 percent of government health expenditure (2). The approval of effective but cost-increasing health technologies and their expanding use have significant opportunity costs, with inevitable impacts on the health budget, as well as profound health implications for patients, families, and the broader community. This highlights the importance of PBAC and MSAC decision making.
Key Criteria to Inform Committee Recommendations
There are alternate ways to characterize the MSAC and PBAC decision-making process. Figure 1 illustrates a simplified three-step sequential algorithm that is commonly used to inform MSAC and PBAC recommendations (Reference Dakin, Devlin, Feng, Rice, O'Neill and Parkin3).
A simplified algorithm of decision making (Adopted from (Reference Dakin, Devlin, Feng, Rice, O'Neill and Parkin3) and revised).

Within the current evaluation process, evidence demonstrating safety, clinical effectiveness, and economic evaluation comprise the core information used to determine whether public subsidy is warranted. As Figure 1A shows, the first step involves the assessment of comparative safety and clinical effectiveness of alternate health technologies. The clinical assessment is central to decision making, because if the technology has no place in clinical practice, it is a moot point as to whether it will be cost-effective. Although PBAC and MSAC consider applications claiming inferior or equal effectiveness, the personal experience of the authors is that proposed health technologies are commonly more effective than current practice. When a health technology is safe and more clinically effective than its comparator, then formal economic evaluation is used to inform funding decisions (Figure 1B).
With the quality-adjusted life-year (QALY) as the preferred unit of health improvement, economic evaluation typically assesses the additional cost of producing an additional unit of health outcome when the new health technology is used instead of its comparator. This is presented in the form of an incremental cost-effectiveness ratio (ICER).
Funding decisions should be informed by making comparisons of the ICER of a new health technology with a threshold or reference ICER that should reflect the health opportunity costs (Reference Edney, Haji Ali Afzali, Cheng and Karnon4;Reference Carter, Vogan and Haji Ali Afzali5). PBAC and MSAC have never made an official statement about the use of a threshold, although it is apparent that industry sponsors target an implicit threshold of AUD50,000/QALY (Reference Wang, Gum and Merlin6). With no empirical foundation to reflect opportunity costs, the current ICER threshold does not necessarily improve population health. The use of an appropriate explicit threshold to inform decisions increases confidence that funding decisions are improving population health and improves transparency in current HTA processes in Australia (Reference Edney, Haji Ali Afzali and Karnon7).
According to guidelines for submitting applications to the two committees (8;9), PBAC and MSAC recommendations can be influenced by a set of other considerations (Figure 1C). Important examples of “other relevant factors” include the following:
• Equity, that is, who receives the health gain, for example, should some priority be given to children, disadvantaged populations, or individuals at the end of life?
• The severity and rarity of the medical condition, for example, priority given to treating sicker patients (expected to lead to premature death) or patients with rare conditions
• Effective therapeutic alternatives, for example, priority given to treating conditions for which there are no existing treatments
Public Input: Problems and Key Implications for HTA Processes in Australia
To make their recommendations, MSAC and PBAC undertake a rigorous appraisal of the presented clinical and economic evidence. Over the past few years, there has been an increasing recognition of the importance of public involvement in HTA, and attempts have been made to involve consumer representatives in both PBAC and MSAC processes. Currently, these committees comprise 1–2 consumer representatives with other members (16-18 members) with technical expertise (e.g., clinical research and health economics). PBAC and MSAC also allow submissions through a short period of online consultation. In 2017, an “HTA Consumer Consultative Committee” (CCC) was established to provide strategic advice and support to the principal HTA committees and the Department of Health (10). Additionally, in 2019, an “HTA Consumer Evidence and Engagement Unit” was established within the Australian Department of Health to investigate the development of projects of consumer and patient engagement (11).
These inputs into the PBAC and MSAC processes, including the CCC and the “Unit” main work plan, primarily focus on the experiences of patients, carers, and patient advocacy groups. Adopting the consensus definitions for these groups provided in Street et al. (Reference Street, Stafinski, Lopes and Menon12), we recognize that the interests of “patients/consumers” are not necessarily aligned with the interest of the public (Reference Menon and Stafinski13). Also, with no specific information in the MSAC and PBAC guidelines to indicate how other factors should be considered within HTA evaluation processes, it is not clear what impact CCC and the “Unit” have on decision making to inform the PBAC or MSAC recommendations.
The HTA system does not formally incorporate public views when assessing health technologies. It is suggested that a formal process is needed to involve the public in HTA in publicly funded healthcare systems, because public funds exist to fund services of value to the public, and HTA processes inform what health technologies should be publicly funded. It is the same rationale for having greater transparency of HTA decision making, that is, tax payer funds are used to determine the societal value of a health technology; therefore, the public should be able to view a submission, its evaluation, and key aspects of the decision-making process, for example, the public summary documents published by the committees.
The inclusion of “other relevant factors” in their submission guidelines indicates that MSAC and PBAC recognize the significance of aspects or dimensions of the value provided by a health technology that are not readily captured through the use of the QALY as a measure of direct health benefits. Notably, depending solely on cost-effectiveness and using the QALY to represent health benefits may discriminate against some patient groups that the public wants to prioritize (e.g., severely ill children), because the new health technologies proposed for the management of these patients may not meet usual benchmarks for cost-effectiveness (Reference Richardson and Schlander14). There is evidence that the public regards such a distinction as critically important for priority setting and wishes to be engaged in healthcare decision making (Reference Richardson and Schlander14;Reference McKie and Richardson15). To address this, MSAC and PBAC guidelines recommend considering “other relevant factors,” such as those listed above. However, if distributional concerns are to be addressed by setting priorities based on principles other than QALY maximization (Reference Marseille and Kahn16), there is a need to systematically define and incorporate public values into the HTA decision-making process. Failure to do so may have a significant impact: for example, national funding bodies may undervalue some health technologies, thereby denying public funding to technologies that the public would consider good value for money.
In Australia, although there is recognition that public value judgments are important to inform funding decisions, the failure to meaningfully and systematically incorporate public values into healthcare decision making has been identified as one of the major limitations of its current HTA system (Reference Wortley, Tong and Howard17).
First, within current HTA evaluation processes, there is no information/guidance as to how influential the “other relevant factors” recommended by the committees are (or should be) when PBAC and MSAC assess the value of new health technologies (Reference Wang, Gum and Merlin6;Reference Whitty and Littlejohns18), for example, how systematically these factors influence funding decisions when ICERs exceed the ICER threshold. Also, there is a lack of guidance on how to include public views in HTA processes (Reference Street, Stafinski, Lopes and Menon12). An analysis of the PBAC and MSAC processes concluded that current HTA decision-making processes would benefit from the systematic incorporation of public perspectives (Reference Whitty and Littlejohns18). Despite some empirical evidence on how the public prioritizes different health gains (Reference Dolan, Shaw, Tsuchiya and Williams19), there is little evidence that public perspectives are used to inform funding decisions. This may be due to uncertainty in the validity of the methods used to elicit public views (e.g., survey or qualitative methods such as focus groups), with evidence suggesting that different public engagement methods lead to different results (Reference Gu, Lancsar, Ghijben, Butler and Donaldson20). Policy makers might question the ability of these methods to produce valid findings on public views when participants have limited information about the problem and hence provide variably informed responses. Another likely factor is the lack of practical guidance on how public views should be systematically and consistently integrated into HTA decision making (Reference Wortley, Tong and Howard17).
Second, with a very limited opportunity for public involvement (e.g., a short period of online consultation), public input is frequently tokenistic and dominated by patient/consumer advocacy group perspectives rather than public views. There is limited empirical evidence that the “other relevant factors” reflect the values of the broader Australian community. Although other committee members are also members of the public, they have a specialist and technical perspective that may not reflect broader public values.
Moving Forward: A Proposal
As decision making is highly context dependent and depends on the skills and experience of the individuals around the table, along with the comprehensiveness and quality of the HTAs before them, there needs to be an approach that explicitly considers public values but in a manner that retains the importance of contextual and consensual decision making. Within the current HTA process in Australia, there remains scope for improvements to explicitly and consistently reflect public values.
One way to address this is via a two-stage process. Stage 1 would involve the use of citizen panels to elicit informed public views on how to determine the value of new technologies to inform funding decisions. A citizen panel builds on the idea that, given enough information about a topic and time for facilitated supported deliberation, a descriptively representative group from the community can collaboratively deliberate with conscience and arrive at a decision that is informed and reflective of broader public values (Reference Murphy21). Citizen panels are increasingly used to elicit public values across a range of public policy areas in Australia, including public funding decision making on assisted reproductive technologies (Reference Hodgetts, Hiller, Street, Carter, Braunack-Mayer and Watt22).
Among HTA national funding bodies, The UK National Institute for Health and Care Excellence (NICE) has been at the forefront of the using a standing Citizens Council to inform perspectives on ethical issues in healthcare policy. Recommendations from a Citizen Council indirectly inform NICE guidance through their inclusion in the NICE “Social Value Judgements” document (23). This document provides important information on values held by the UK public that NICE should follow to align its decision making with public values. However, it does not offer explicit guidance on how to apply these values to decision making. In their study evaluating the Citizens Council of NICE, Davis et al. (Reference Davies, Wetherell, Barnett and Seymour-Smith24) found support and willingness from citizens to play an active role in the panel to inform HTA processes, though a number of obstacles (e.g., challenges in sustaining it over time because of the expense and time required to elicit public values) were also mentioned.
A key output of Stage 1 would be a comprehensive set of values that an informed and representative sample of the Australian public regards as important in making health funding decisions, together with an in-depth understanding of how these compare in terms of their relative importance (e.g., “highly important,” “important,” or “somewhat important”).
Despite some significant evidence indicating that citizen panels offer a feasible method to involving the public in the HTA decision-making process (Reference Menon and Stafinski13), there are a number of limitations associated with this approach, including concerns about the generalizability of the findings. To address this, every effort should be made to recruit participants to ensure that the panel is “descriptively representative” of the Australian public in terms of key characteristics such as age, gender, ethnicity, education, and employment status. Other limitations include difficulties in the active and effective participation of the members of the public in informed discussions involving technical language. This highlights the importance of information materials using appropriate methods to present information to ensure that participants understand the nature of the decision to effectively participate in deliberation and the role of facilitation/moderation to ensure an equal chance to contribute to discussion (e.g., whether the facilitation/moderation is appropriate to encourage balanced participation).
Stage 2 would aim to maximize the practical relevance and policy translation of the Stage 1 findings. This could be achieved by using an expert roundtable comprising HTA key stakeholders (e.g., PBAC/MSAC members, the Australian Government Department of Health, HTA agencies, and representatives of the industry and consumers). The key output of this stage would be an implementation plan for how to incorporate the public values into HTA decision making.
Once a plan has been developed and actioned, it is likely that periodic citizen panels would need to be convened over time to reflect changing values in the community, particularly when disruptive technologies are introduced and expectations are reset regarding what health care should be provided.
Conclusion
Over the past few years, important concerns have emerged about the lack of systematic and explicit incorporation of public views into HTA evaluation processes to inform funding decisions (Reference O'Rourke, Werko, Merlin, Huang and Schuller25). The public may attribute more value to an expenditure based on equity, for example, one that may secure fewer QALYs but directs these to the people with the highest need in a specific context (e.g., people living in remote areas). Although our focus is on the Australian decision-making process, the discussion can be applied more widely to some other national funding bodies that use HTA to inform funding decisions. We propose the use of a deliberative process such as citizen panels as one potential approach to identify and incorporate public values within the HTA process. This proposal would help ensure that government spending on new technologies achieves value for money as assessed by both scientific experts and the public.
Acknowledgment
No specific funding has been provided for the preparation of this manuscript.
Conflicts of Interest
HHAA has served as a member of the Evaluation Subcommittee (ESC) of the Medical Services Advisory Committee (MSAC) since 2015. JS has nothing to declare. TM is a Director of the Adelaide Health Technology Assessment (AHTA) and has conducted HTA since 2001. JK has served as a member of the Economics Subcommittee (ESC) of the Pharmaceutical Benefits Advisory Committee (PBAC) since 2009.

