Background

Poland’s reimbursement legislation, enacted as the Law on Health Care Services Funded from Public Sources (Reference del Llano-Señarís2004) and the Reimbursement Act (Reference Drummond, Schwartz, Jönsson, Luce, Neumann, Siebert and Sullivan2011), and further specified by Ministerial ordinances (Ordinance on Minimum Requirements Reference Plisko2012), stipulates that every application for reimbursement of a medicine for a particular indication must be supported by a set of HTA analyses, covering clinical, economic, budgetary impact and rationalisation issues (the last comprising measures to mitigate any additional budgetary expenditure). The manufacturer submits these analyses, typically compiled by consultancy firms, to the Ministry of Health (MoH), which is then supposed to pass them ‘without any undue delay’ to the AHTAPol. Once received, AHTAPol’s analysts develop a VA, which is then forwarded to the Transparency Council (TC), comprising senior health care experts appointed by the Minister of Health. Reports from TC sessions suggest that VAs constitute the primary source of evidence considered by the TC, with contributions from clinical experts or patient organisations being less prominent. The opinion of the TC as to whether a drug should be reimbursed or not is then considered by the President of the AHTAPol, together with the VA. The President issues a recommendation to the Minister of Health, which may differ from the TC opinion. Subsequently, the VA, the TC opinion and the President’s recommendation are passed to the MoH and become the basis of price negotiations with the manufacturer, led by the Economic Committee at the MoH and subject to the Minister’s final decision.

While the ATHAPol receives core public funding, the cost of developing a VA is covered by the manufacturer, with the upper limit of 150,000 PLN (£30,000). The Minister has 180 days to reach a decision on a new reimbursement application, including 60 days during which the AHTAPol President must reach a recommendation after receiving the reimbursement application from the MoH.

The transparency of VAs is subject to several regulations that may, in some circumstances, conflict. The AHTAPol has a duty, introduced by the reimbursement legislation, to make VAs publicly available on the internet, together with the corresponding manufacturers’ HTA analyses and any comments on VAs, including, for example, clarifications by the manufacturer. Publication should take place at least 8 days before the corresponding TC session (AHTAPol 2015). However, the AHTAPol is also subject to the Law on Unfair Competition (Reference Bendor, Glazer and Hammond1993), which states that revealing an enterprise’s ‘commercial secrets’ constitutes unfair competition. The Law on Protection of Personal Information (Reference Bochenek, Kocot, Rodzinka, Godman, Maciejewska, Kamal and Pilc1997) also requires that processing personal data, including its publication, must be consensual and ‘necessary’ for meeting legal obligations of an institution or advancing the public interest.

The AHTAPol has gradually clarified these conflicting regulatory requirements in relation to VAs (Bochenek et al., Reference Kaltenthaler, Papaioannou, Boland and Dickson2016). An initial communication emphasised that the transparency of public information may only be limited in exceptional circumstances and demanded that manufacturers’ requests for redacting information from VAs be clearly justified with respect to its ‘economic value’ (AHTAPol, Reference Gulácsi, Rotar, Niewada, Löblová, Rencz, Petrova, Boncz and Klazinga2012). A subsequent communication stated that the AHTAPol would only redact VAs in response to requests from the manufactures (AHTAPol, Reference Hailey2014). The only pieces of information that the AHTAPol would redact on its own initiative would be those ‘directly’ concerning risk-sharing instruments and prices proposed by drug manufacturers, an important innovation brought in by the Reimbursement Act, seeking to manage the budgetary impact and prices of new medicines by relating them to their health effects or volume of sales (Kawalec et al., 2016). By contrast, the AHTAPol would publish all other information – in particular, data on clinical effectiveness, safety, cost-effectiveness, budgetary impact and reimbursement arrangements in other countries. Most of these measures were broadly supported by HTA consultants (Jakubiak, Reference Löblová2014), with more scepticism shown by the industry, highlighting the need for further refinement (INFARMA and IGFP, 2014).

As already noted, the AHTAPol’s drug appraisals are broadly similar to NICE’s STA, first undertaken in 2005 (Kaltenthaler et al., Reference Meneu2011). Similar to the AHTAPol’s HTA process, STAs concern mostly new technologies for a single indication based primarily on clinical and economic evidence submitted by manufacturers. In contrast to the AHTAPol, NICE outsources the development of assessment reports to ERGs, independent external academic centres (NICE, 2017). While in STAs the appraisal phase is also led by senior health experts, it puts greater emphasis on consultation with patients, clinicians and the industry (NICE, 2014). Another difference between them is that NICE provides only indicative timelines for various stages of the STA process, including the development of ERG reports (NICE, 2009). Finally, unlike AHTAPol’s advisory recommendations, those developed by NICE must be implemented by health commissioners within a specified period.

NICE considers public availability of evidence as a vital form of transparency (NICE 2014). In addition to the types of documents published by the AHTAPol, NICE also publishes, in their entirety, comments that its consultees provide to the Appraisal Committee. However, neither organisation publishes expert opinions that inform the development of assessment reports. NICE only accepts redacted information ‘in exceptional circumstances’. First, like the ATHAPol, it recognises that some evidence may be ‘commercial in confidence’, with potentially ‘a significant impact on the commercial interests of a particular company’. Second, unlike the AHTAPol, NICE uses the category of information ‘academic in confidence’, with potential implications for ‘the ability of the data owner to publish the information in a scientific paper’ (NICE, 2014). Informed by agreement with industry stakeholders reached in 2004, NICE has produced extensive guidance on handling various forms of confidential information (NICE, 2014, n.d.).

Findings

We begin this section by evaluating changes in the transparency of the HTA process and outputs in the light of AHTAPol’s VAs. We then compare these findings with exploratory analysis of NICE’s ERG reports.

AHTAPol: transparency of HTA process

Rationale for withdrawing information

The AHTAPol met its duty to publish VAs in 98.8% of cases, with just two unavailable on its website (70/2012 and 54/2013). Their absence was not accounted for.

All published VAs had some information redacted. In 19 instances, most from 2012, the AHTAPol provided no reasons for redacting information (e.g. 71/2012, 42/2013). Prior to VA 103/2013 the AHTAPol only provided two general reasons for blackouts, either protecting ‘commercial secrets of the applicant’ or ‘personal data and commercial secrets belonging to other commercial entities’. In the latter case, however, the categories of people or other commercial entities were not specified; presumably the AHTAPol referred to HTA consulting firms preparing the analyses on behalf of the manufacturers. The distinction between the two categories of exclusions was supposed to be reflected by using different colours in the main body of the VA, but this was never applied in practice in this period.

Starting from VA 53/2013, and then consistently from VA 104/2013, the AHTAPol reformulated the reasons for withdrawing information as ‘protection of commercial secrets of the commercial entity’ and ‘protection of privacy of physical persons’, tying them to specific legal grounds, namely the Law on Unfair Competition and the Law on Protection of Personal Information. In practice, the AHTAPol applied the principle of protection of commercial secrets to research from manufacturers’ HTA analyses cited in the VAs; in some instances this principle was extended even to the name of the manufacturer (e.g. 66/2014). While the AHTAPol stated that redactions were made in the interest of drug companies, they also included the names of other entities, most likely HTA consultancy firms. The AHTAPol used the principle of protection of personal data in relation to external clinical experts (e.g. 191/14), patient organisations (e.g. 50/2014) and authors of manufactures’ HTA analyses (e.g. 259/2013).

While VAs stated that the AHTAPol undertook redactions, they did not provide details of the decision-making process, including who could make such requests, how they were processed, on what criteria they were based and who decided.

Authors and contributors

All VAs since 2012 included a section that could list AHTAPol employees, external clinical experts and other contributors involved in, for example, analysing epidemiologic, clinical economic data or risk-sharing instruments (e.g. 86/2012). This section was gradually withdrawn throughout 2013 and disappeared completely in subsequent years. Nevertheless, even when it was included, names of all AHTAPol employees and external experts were always redacted, and only in one case another contributor, a medical specialty organisation, was named (29/2012).

VA timeline

All VAs reported when they were completed. However, as shown in Table 3, up to 24.5% of VAs in subsequent years provided only a monthly date of completion, with the percentage of those providing daily dates of completion reaching up to the 86.9% in 2012. Further, the overwhelming majority of VAs included the daily date of receiving the manufacturer’s reimbursement application from the MoH, thereby signifying the formal beginning of the process of developing a VA. Only in one instance (98/2012) the date was blacked out and in nine other instances it was not provided. However, VAs reported the statutory deadline for returning documents to the MoH less frequently over time, with the percentage falling from 16.4% in 2012 to 2.3% in 2013.

Table 3

Key milestones in the development of verification analyses

An even smaller, and declining, percentage of VAs provided both the date of receiving the reimbursement application from the MoH and the statutory deadline for returning the recommendation of the AHTAPol President to the MoH, following the completion of the VA. Among this group, only four stated that the deadline was 60 days, the statutory figure. In all other instances it was considerably longer. While there is a proviso that the statutory deadline can be extended, no explanations were provided for any extensions.

Subject of reimbursement application

Only in one case was the name of the medical technology applying for state reimbursement redacted (119/2014) and in 15 instances the indications of medicines were redacted partially or entirely (one in 2012, seven in both 2014 and 2015). The redactions, however, ceased in 2015. With two exceptions (56/2014 and 105/2014), all VAs included a section specifying the reimbursement scheme targeted by the application. Nevertheless, we identified 28 examples issued before 2015 in which the name of reimbursement scheme was removed (e.g. 223/2013).

As detailed in Table 4, with seven exceptions from before 2015, all VAs included a section on possible risk-sharing instruments. Information on the inclusion of these agreements in the manufacturer’s application was initially redacted, with as much as 91% of the relevant sections being blacked out in 2013. While these redactions were eliminated in 2015, even general types of risk-sharing instruments were never mentioned in any of the VAs.

Table 4

Key types of information on the subject of reimbursement applications included in verification analyses

Manufacturers’ HTA analyses

Only with one exception (139/2014), all VAs had a section to note the receipt of the four types of HTA analyses (decisional, economic, budget impact and rationalisation) that had to support reimbursement applications. Over time, there was a decline in the share of VAs reporting the receipt of all HTA analyses, from 41% in 2012 to 34% in 2015. In eight cases before 2015, the relevant section was included but information on the receipt of all HTA analyses was redacted. There was, however, a slight increase in the share of HTA analyses whose titles and organisational authors were visible (from 26 to 30% and from 3 to 14%, respectively).

Expert opinions

As Table 5 shows, in 2012, the share of VAs stating the number of experts that the AHTAPol had approached for an opinion and who responded to requests was very high (98.4 and 91.8%, respectively). Following the removal of the table listing contributors to the VAs (see the section on authors and contributors above), this share declined in 2013 and 2014, and then picked up again, reaching 38.4 and 61.6%, respectively, in 2015. While almost all VAs had a separate table summarising expert opinions, it did not normally provide information on the number of experts approached and responding to AHTAPol’s request for opinion. Further, the share of VAs with the summary table dropped to 91.9% in 2015.

Table 5

Requested, provided and approved expert opinions in verification analyses

The removal of the table listing contributors had a negative impact on the transparency of data on expert opinions. As Table 6 demonstrates, while in 2012 91.8% VAs clearly stated both the number of experts approached and providing opinions, this share dropped to 37.2% in 2015. Similarly, the share of VAs that included both the number of experts providing opinions and a summary of expert opinions dropped from 91.8% in 2012 to 58.1% in 2015.

Table 6

Relationships between requested, provided and approved expert opinions in verification analyses

These problems are illustrated by inconsistencies between expert numbers listed in the summary table and those referred to throughout VAs. For example, VA 114/2015 initially stated that one expert opinion had been received but the summary table listed no opinions. Conversely, VA 91/2015 seemed to suggest receiving three expert opinions but four appeared in the summary table.

The inconsistency between the number of experts approached and providing opinions was, in some instances, explained by referring to experts’ failure to respond to AHTAPol’s request but in others it was not commented on. Further, inconsistencies between the number of opinions received and included in VAs can, only to a small extent, be explained by information on experts’ conflicts of interest. While there was an increase in the share of VAs in which some expert opinions were excluded (from 13.1% in 2012 to 40.5% in 2015), the share of VAs attributing exclusions explicitly to conflicts of interest was markedly lower (1.6% in 2012 and 11.6% in 2015). Even in these instances, however, the occurrence of conflicts of interests was merely acknowledged, with no further details provided. The remaining reasons for excluding expert opinions were not accounted for.

The VAs did not provide any information on the rules for approaching experts, and possible exclusions of expert opinions. Only VAs 75 and 76/2015 reported that it was the President that made decisions as to whether expert opinions with conflicts of interest could be accepted.

Finally, as Table 7 details, the share of VAs with at least one expert name redacted fell from 100% in 2012 to 67% in 2015. The percentage of VAs containing at least one expert opinion redacted also fell from 18% in 2012 to 0 in 2015. Further, the average percentage of redacted expert names in VAs dropped from 100% in 2012 to 50% in 2015. Likewise, the average percentage of redacted expert opinions in VAs fell from 6.5% in 2012 to 0% in 2015. It was never explained, however, why experts’ names or opinions were redacted or not.

Table 7

Verification analyses with redacted expert names or opinions

Note: Verification analyses with no opinions in the table summarising expert opinions were excluded from the counts.

AHTAPol: transparency of HTA outputs

Transparency of VA conclusions

Each VA had a section summarising its conclusions. As presented in Table 8, in 2012, the share of key conclusions affected by redactions was very high. For example, all sections on ‘Relation between costs and health effects and threshold price’ were entirely redacted. The only exception to this rule was for the section describing the medical technology, where there were no redactions. The share of redacted key conclusions was increasing in most areas in 2013 but then dropped considerably in 2014 and 2015. For instance, the share of redacted conclusions on alternative medical technologies dropped from 26.7% in 2012 to 1.2% in 2015. Nevertheless, even in 2015 the percentage of redacted conclusions on cost-effectiveness, budgetary impact or risk-sharing instruments remained high (76.7, 67.4 and 50%, respectively). These redactions typically make it difficult to understand whether the drug applying for reimbursement was more or less cost-effective than its comparators, what the projected size of the budgetary impact was and how budgetary risks would be shared between the manufacturer and the public payer.

Table 8

Redactions in key conclusions reported in verification analyses

Note: Given changes in the format of verification analyses, the denominator for calculating the percentages in the table was the number of verification analyses with a particular section, and not the total number of verification analyses in a given year.

NICE: transparency of HTA process and outputs

Starting with the transparency of the HTA process, although some ERG reports mention the two main rationales for redacting information, ‘commercial in confidence’ and ‘academic in confidence’ in their initial section, this was not the case in the analysed sample. The appearance of some of the ERG reports suggests that these principles had been applied in the form of different colours at earlier stages of the STA process and are then replaced with uniform black colour before publication.

All ERG reports provided a list of authors, and all but one detailed their tasks. Contributors, including external clinical experts, were mentioned in the acknowledgements section. In two cases, we found redactions applied to organisational affiliations. With one exception, ERG reports had a section to include any conflicts of interest reported by authors. Any contributor’s conflicts of interests were either included in the section mentioned above or linked with acknowledgements. In both cases, specific conflicts of interest were mentioned. However, the absence of a separate conflict of interest section for contributors could create ambiguity.

All but one ERG report stated a date of completion. In two cases, this was a monthly, rather than a daily, date. The ERG reports did not state when they commenced, providing instead a project number.

All ERG reports stated the name of the drug, its indication and characterised the target patient population. As any special reimbursement arrangements, called patient access schemes, could be negotiated between the manufacturer and the Department of Health during later stages of the STA process, they were mentioned in the final guidance documents. The guidance described the general nature of these arrangements without revealing the size of discounts from the official list price (which was publicly available).

All ERG reports characterised the manufacturer’s submission in the summary chapters, with the company always being listed as the author. Nevertheless, only in five instances the manufacturer’s submission was included in the reference list.

As mentioned above, ERG reports did not have a separate section to list clinical experts, nor did they state how many experts had been approached. In four cases, authors were mentioned as sources of expert advice. Nevertheless, in 14 cases clinical experts were included in the ‘contributor’ category, sometimes together with, for example, people providing other forms of support. In two cases, both authors and contributors were listed as sources of clinical advice.

ERG reports did not have a separate section summarising expert opinions. Rather, the opinions were summarised in text in relation to a specific issue, potentially obscuring any divergence of opinions between experts. Although opinions were never attributed to specific individuals in some reports, with only one expert, it was possible to establish their identity. Notably, in two cases, even though the expert advice had been provided, it did not seem to be included in the main body of the report.

Regarding the transparency of HTA outputs, 15 out of 22 ERG reports had no redactions in the summary section, as demonstrated in Table 9. While the number of reports affected by redactions appeared to be increasing over time, this finding must be taken with scepticism given the small sample size.

Table 9

Redactions in Evidence Review Group reports

Conclusions and discussion

To assess the extent to which the normative ideal of transparency of HTA is achieved in the real world, we conducted a comprehensive analysis of the HTA process and outputs in Poland since 2012, and compared it with the gold standard of transparency, NICE, as exemplified by its STA process. Our analysis suggests that, overall, the AHTAPol meets or outperforms NICE’s transparency levels in several areas. However, there are also points where AHTAPol’s transparency lies behind NICE’s, and areas where both agencies could improve.

The AHTAPol seems to have reached the standards set out by NICE in transparency of HTA outputs. The decreasing share of VAs with redacted conclusions regarding the nature of the health problem, description of the medical technology, its clinical effectiveness and safety is a clear sign of improvement. Perhaps surprisingly, our stratified sample suggests that the key conclusions presented in NICE’s ERG reports may be demonstrating an opposite tendency towards decreasing transparency. The VAs have also demonstrated progress, albeit less decisive, in key conclusions regarding cost-effectiveness, budgetary impact and risk-sharing instruments, where redactions remain common (at least 50% of VAs). It is notably debatable whether the mere fact of reporting the inclusion of risk-sharing instruments represented an increase in transparency, bearing in mind that the VAs fell short of providing even a general description of risk-sharing instruments listed in the reimbursement legislation, let alone any potential specific solutions that might have been proposed by manufacturers. Beyond this point, however, our findings regarding HTA outputs are consistent with those of Bochenek et al. (Reference Kaltenthaler, Papaioannou, Boland and Dickson2016), who reported radical improvement in the transparency of AHTAPol recommendations in the same time period, following a clarification of AHTAPol’s policy in 2014 on dealing with manufacturers’ requests for introducing redactions to HTA outputs (AHTAPol, Reference Hailey2014).

The AHTAPol seems to perform just as well as, if not better than, NICE in certain aspects of the HTA process. It systematically publishes its assessment reports, VAs and provides key types of information on reimbursement applications, including clinical indications and the names of reimbursement schemes. The AHTAPol seems to outperform NICE in providing reasons for redactions and applying them systematically in assessment reports. VAs also acknowledged that manufacturers’ HTA analyses might have been outsourced to consultants. Finally, the AHTAPol did a better job at reporting the beginning of the assessment process systematically (not included in ERG reports) and, even more importantly, providing tables summarising expert opinions.

There were, however, important areas of the HTA process in which the AHTAPol underperformed against NICE. The baseline transparency related to authors, clinical experts and other contributors was very low (all names redacted in 2012) and decreased even further over time with the disappearance of the relevant section from VAs. While the removal of information on external experts was somewhat compensated for by a separate section summarising expert opinions, the same cannot be said for other contributors. It remained unclear, therefore, to what extent the AHTAPol relied on the assistance of, for example, consultants in developing VAs. Crucially, unlike NICE, the ATHAPol merely reported (unsystematically) the mere fact of reporting a conflict of interest by an expert, without mentioning its nature, extent or commercial interests involved. Further, no information was provided on AHTAPol’s processes for managing conflicts of interest. Inevitably, this diminished the accountability of decisions regarding the exclusion – or admission – of expert opinions associated with conflicts of interest.

There were also problematic areas related to the unique design of the HTA process in Poland, with no direct comparison with the STA process. The AHTAPol lacked transparency in providing information on the beginning, statutory deadline and effective completion of the assessment procedures. By 2015 it had practically stopped reporting of the statutory deadline for returning completed recommendations by its President. Consequently, it became less clear how much time the analysts, TC and President had available to, respectively, develop a VA, issue an opinion and recommendation. The lack of information on the statutory deadline could obscure instances of delayed transfer of documents from the MoH, which may exert additional pressure on the AHTAPol to process manufacturer’s submission to meet this deadline. Further, the absence of information on the statutory deadline made it impossible to establish whether it had indeed been observed. This seems to be a less pressing problem for NICE as the timelines for completion of subsequent stages of the STA process are only indicative (NICE, 2009).

Lastly, reporting of expert opinion in the HTA process is a key area for improvement for both the AHTAPol and NICE. The AHTAPol showed diminishing transparency in reporting the number of experts approached, providing opinion and those whose opinions were included in VAs. The absence of this information makes it impossible to evaluate the extent of non-responses in expert opinions and, even more importantly, the instances of reporting conflicts of interest and their relationship with excluding expert submissions by the AHTAPol. By reporting non-attributable summaries of expert opinions, NICE’s ERG reports also obfuscate potential conflicts of interests of contributing experts as well as possible scientific dissensus.

Overall, the lack of decisive improvement in the transparency of the HTA process in Poland might be explained with reference to the fact this area was not covered explicitly by the revised AHTAPol’s policy on dealing with manufacturers’ requests for redactions, focusing on the transparency of HTA outputs. This might have created a procedural loophole negatively affecting the transparency of AHTAPol’s work.

More broadly, the challenges to transparency identified in AHTAPol’s VAs, and to some extent in NICE’s STA process, highlight two major concerns, familiar to students of delegation of authority to non-majoritarian institutions: the risk of their capture by private interests, and the uncertainty about the quality and efficiency of their work (Bendor et al., Reference Jakubiak2001; Pollitt et al., Reference Sorenson, Drummond and Kanavos2001; Thatcher, Reference Nizankowski and Wilk2002). To prevent risks and suspicions of regulatory capture, HTA bodies (including NICE and the AHTAPol) introduce rules on conflicts of interest. Even the most stringent of these rules, however, lose their function as an indicator of legitimacy and accountability to the public interest if their application lacks transparency. It is for this reason that the AHTAPol’s limited reporting of the authors, experts and contributors to its VAs is concerning, and that the practice of collating expert opinion in NICE’s ERG reports deserves rethinking. The Polish case calls for particular caution as the gaps in reporting expert opinions and conflicts of interest may be part of a broader set of challenges associated with widespread relationships between senior clinicians advising the AHTAPol and the pharmaceutical industry (Ozierański et al., Reference Pollitt, Bathgate, Caulfield, Smullen and Talbot2012a; Ozierański and King, Reference Pollitt, Bathgate, Caulfield, Smullen and Talbot2017). The absence of information on contributions from, and potential conflicts of interests of, AHTAPol employees involved in developing VAs is especially puzzling given the well-documented revolving door syndrome between the ATHAPol and the commercial sector, including both consultancies and drug companies (Ozierański and King, 2016).

The non-existent or imperfect reporting of relevant timelines gives, in turn, reason for concern as transparent timelines provide a quick (if crude) indicator of the quality and efficiency of the body’s work. While sufficient time to develop HTA assessment reports does not automatically guarantee its quality, excessively tight deadlines might point to a problem of quality or efficiency. Extremely short appraisal times could indicate that HTA is merely a ‘tick-the-box’ exercise without meaningful scientific input into pricing and reimbursement decision-making. Extremely long timelines, in contrast, would point to inefficiencies within the agency. Either of the two alternatives calls into question the relationship between principal and the agent. Blurred accountability lines between the MoH and the AHTAPol have been well described in Poland (Ozierański et al., 2012a, Reference Pollitt, Bathgate, Caulfield, Smullen and Talbot2012b). Our findings regarding the insufficient reporting of HTA timelines by AHTAPol indicate a potential area for exercising pressure from ministerial decision-makers to expedite appraisals.

Our study has two main limitations. One is the use of only a stratified sample of NICE’s ERG reports. While a full analysis of ERG reports was not deemed necessary because of NICE’s established status as a gold standard in transparency of HTA, our stratified sample suggests that full examination by future researchers might be warranted. Another limitation in our examination of HTA outputs is that it did not quantify the extent of redactions and rate the extent to which they may affect our understanding of the key conclusions (Bochenek et al., Reference Kaltenthaler, Papaioannou, Boland and Dickson2016). Nevertheless, even without using these techniques it was possible to investigate what transparency of HTA means in practice, as well as to test our key expectation regarding its improvement over time. Therefore, we contributed to the growing knowledge of Polish HTA, an important learning case for other middle-income countries, especially in the CEE region, which are in the process of establishing their HTA systems (e.g. Bulgaria or Romania). We also added to the emerging literature on comparative transparency of HTA by developing easily interpretable comparisons with NICE and pointing to areas for improvement for both bodies.

Recommendations

Our findings allow for formulating several policy recommendations for the AHTAPol.

1. 1. Increase the clarity of decisions regarding the selective application of redactions to, for example, expert names or contributions.

2. 2. Reintroduce the section detailing authors and contributors at the beginning of each VA. It should include the number of experts approached, those providing opinions and those whose opinions have been included in the assessment process, as well as specific reasons for excluding expert opinions.

3. 3. Ensure that all VAs include information on statutory deadlines for returning VAs, TC’s opinions and President’s recommendations to the MoH. Any delays should be clearly explained.