Introduction
Canada's Food and Drugs Act defines medical devices as, “a wide range of health or medical instruments used in the treatment, mitigation, diagnosis or prevention of a disease or abnormal physical condition.” In accordance with the Act and Medical Devices Regulations, Health Canada assesses medical devices for their clinical effectiveness, safety, and quality for license approval in Canada, and it provides oversight to ensure that safety risks of devices are identified and mitigated (1).
The Health Products and Food Branch (HPFB) at Health Canada is undertaking the Regulatory Reviews for Drugs and Devices (R2D2) project with a focus on access improvement for both drugs and devices. One objective related to this focus is to determine the anticipated and ongoing real-world data and evidence (RWD and RWE) sources across Canada to help modernize medical device assessment and surveillance and to make regulatory decisions and health technology assessment (HTA) recommendations throughout the product life cycle (2). More specifically, as a result of use in broader populations, for longer durations, or under different conditions of use, new information related to the benefits and risks of a medical device can be obtained (3).
To better understand the nonregulatory Canadian landscape of RWD sources for medical devices, Health Canada carried out an environmental scan to identify existing and planned RWD sources across the country related to the use of medical devices. One component of the scan involved key informant interviews with different stakeholders, including government decision makers, clinicians, HTA producers, researchers, industry, and patients, to understand their experiences with and their perspective on how to enhance the use of RWE for medical devices. As well, a national survey and the literature review on the RWD sources and uses of RWE for medical devices were performed for the scan, and their results are published elsewhere (4). Results from the interviews as presented in this paper were collected as part of the environmental scan.
For our study, RWD was defined as data relating to patient status and/or the delivery of healthcare routinely collected from a variety of sources, and RWE refers to the evidence regarding the usage, and potential benefits or risks, of a medical product derived from analysis of RWD (5).
Methods
Approach
A grounded qualitative approach was used to gather, synthesize, analyze, and conceptualize data from stakeholders and to identify opportunities for national and international collaborations on the current uses for RWD collected and RWE generated (Reference Strauss and Corbin6;Reference Butterfield, Borgen, Amundson and Maglio7). Participants were clinicians, health researchers, biomedical and clinical engineers, decision makers in the regional and provincial health authorities, HTA producers, and industry. Another set of questions were created for patient representatives, where the investigators inquired about their perspectives and expectations of RWE. Telephone interviews of approximately 30–45 min were audio-recorded and transcribed verbatim.
Sample Selection and Recruitment
For the interview sample, the investigators invited forty-seven individuals. In qualitative research, information from diverse perspectives, and not a large sample, is required. The interviewees were identified by Health Canada and their network. Prospective participants were invited to participate via email and were notified that neither they nor their institution would be disclosed in any public-facing documents. Sampling was concurrent with the data collection and analysis and ceased when no new unique key concepts or themes emerged. Thematic saturation, where the categories were explained thoroughly and new categories did not emerge, was determined upon review of the codes derived from each interview response.
Data Collection
Data collection and analysis followed the principles of grounded qualitative research (Reference Pope, Ziebland and Mays8;Reference Auerbach and Silverstain9). An interview guide in English and French was created to ask the interviewees about their experience and expectations with and understanding of RWE (Supplementary File 1). Before starting each interview, we ensured the participants’ anonymity and confidentiality and reminded them that their responses would be audio-recorded.
Data Analysis
One reviewer (JP) independently reviewed the transcribed interviews to identify, define, and organize key concepts in an inductive manner, and another reviewer (VN) verified the results of the qualitative analysis. A log was maintained to capture all the codes, their definitions, and examples from the interviews to illustrate their application. Using a constant comparative technique, the narratives from the interviews were compared and contrasted to determine if the concepts should be expanded or merged.
Results
Participants
Among the forty-seven stakeholders and patient representatives invited to participate in our telephone interviews, two declined and eight did not respond to our invitation. Three individuals initially accepted our invitations, but then declined due to unforeseen circumstances. We conducted and recorded thirty-four interviews with forty individuals as six participants requested that one of their colleagues also be part of the interview. Almost half of the interviewees (17/40; 42.5 percent) represented pan-Canadian organizations, and the province of Alberta has the largest representation in our sample (7/40; 17.5 percent). Other provinces represented were Ontario (5/40; 12.5 percent), Quebec (4/40; 10 percent), Manitoba (2/40; 5 percent), and Nova Scotia 1/40; 2.5 percent). International perspectives included the United States (3/40; 7.5 percent) and United Kingdom (1/40; 2.5 percent) (Table 1). In terms of sector, seven participants (17.5 percent) were from the industry, and five each represented academia, clinical sector, nongovernmental organizations, and patient groups.
Characteristics of interview participants
Interviews
Results of the interviews are presented by key concepts as follows: (i) current RWE use for medical devices; (ii) future aims and use of RWE; (iii) organizational aspects of RWD collection; (iv) medical and scientific aspects of RWD collection and RWE generation; (v) privacy issues and legal aspects of RWD collection; and (vi) options for implementing the use of RWD and RWE.
Current RWE Use for Medical Devices
Data and information derived from “real-world” settings can help to inform regulatory assessments, post-market surveillance (PMS), and medical device enhancements and innovations. Some participants use RWE for research purposes and to understand clinical trials. As well, RWE can facilitate the investigation of causes of device malfunctions, identify trends in health service delivery and unmet medical needs, and understand the accessibility and appropriate use of medical devices:
“Pre-market research real-world data has proven that it can be very useful to identify problems with the technology among patients who are not included in the randomized control trial. Patients [in the real care setting could] be older, have more comorbidities, or be in a lower socioeconomic status. With medical devices, you need to consider the user of the device. In pre-market research, the user tends to be highly specialized on the topic and highly trained. In routine practice, the user of the device may not be an expert, and the RWD would any issues or problems associated with the use of this device” (02 academic).
Furthermore, RWE is used to conduct HTAs, inform HTA recommendations on medical devices, and advise the provincial government on capital expansion and expenditure.
The data and information collected are also used to monitor failure rates of implants, assess if clinicians meet the standard set by the jurisdiction, provide feedback to clinicians and healthcare decision makers to improve health services performance, and determine the appropriate time to replace medical equipment.
Future Aims and Use of RWE
Pre-market testing and the device usage in real-world settings are not always aligned. This disconnect, therefore, makes it a challenge to investigate the cause of and resolve potential problems. RWE use can, therefore, elucidate the requirements for medical development and testing prior to market entry. As a result, the surveillance methods and processes for different medical device-types facilitated with the use of RWE can inform both regulatory and HTA bodies and narrow the gap between pre- and post-market requirements:
“For regulation, we feel that the data should be informing how the devices are licensed before they're brought onto the market. For some devices, the real-world usage of the device, for example, infusion pumps does not match the data collected from the testing. For infusion pumps, the fluids used for testing (e.g., water) are not the same as those used in real life (i.e., medication). The actual usage of the device does not match the pre-market testing of the device. So there is a disconnect with the device and it is hard to resolve the problem by simply modifying the device without testing it using real life situations using different thresholds. As we start to understand how the device is used in real patients in real health care delivery, we can start to understand how the devices should be developed and tested prior to market access” (01 government).
In addition, the application of RWE can help to determine if a clinical trial is necessary to inform a decision, and identify relevant patient populations and outcomes in future clinical trials. A participant spoke about the need to facilitate access in some jurisdictions, including Canada, to a new health technology at an earlier stage, so more inhabitants can participate in international research studies. The use of RWE is also viewed as a means to reduce the cost of conducting clinical trials and burden of data entry. For example, a participant stated that automated data collection in electronic health records can increase clinical engagement as they will not realize they are collecting RWD for RWE generation. Another participant believed that regulatory bodies, like Health Canada, should carry out more outreach to the clinical community to raise awareness of the regulatory process, so clinicians have greater clarity on the importance of data collection in their routine practice.
From an HTA perspective, the use of RWE can increase the accuracy of cost-effectiveness analyses, as well as enable inclusion of the local context to support reimbursement decision making. The data and information can also be applied to assess public funding available versus out-of-pocket payments to deliver patient care.
Organizational Aspects of RWD Collection and RWE Generation
Leadership is necessary to have various stakeholders onboard. National leaders in their medical specialty areas can help to raise awareness of RWD collection with their medical community. Some participants have indicated that regulatory and HTA bodies must explicitly communicate the purposes for which RWD will be collected and how RWE generated from these data will be used for regulatory decisions and HTA recommendations:
“For regulatory decisions, a clear line from [the regulator] Health Canada indicating we're going to use the data for these purposes needs to happen. And a statement from HTA bodies indicating that we will use this information for reimbursement decisions needs to happen, too. Nobody will collect the data unless it's clear from both [the regulator] Health Canada and HTA bodies that this data will be used” (01 HTA agency).
Participants also expressed a desire for regulatory bodies to play a leadership role to improve data access across jurisdictions and provide guidance on RWE use that describes what is acceptable for submissions. For instance, numerous participants in Canada suggested that the government incentivize industry to follow the new reporting regulations, Protecting Canadians from Unsafe Drugs Act (Vanessa's Law), to further strengthen the safety and surveillance of drugs and medical devices that came into force in 2019 (10).
Medical and Scientific Aspects of RWD Collection
For data collection to be successful, participants have stated that a collection of the willing is necessary. This would involve a group of individuals and organizations who agree on the RWD to be collected and supported by the regulator and HTA bodies. Data collection would happen in actual clinical practice, so healthcare providers should always be involved to provide insight on what data are important to collect. It is, therefore, paramount to have their buy-in and input.
RWD is representative of routine practice, and the patient population is representative of those who receive care in routine practice. The heterogeneity of outcomes also increases the generalizability for decision making. Furthermore, RWD can help stakeholders to understand proficiencies, volume–outcome relationship, and early learning curves associated with the use of medical devices:
“It depends on what data you're trying to collect and for what purpose as who should be involved. Everybody has a role to play, and they vary between one another” (05 clinician).
One participant stated that as RWD can introduce biases in observational data for comparative effectiveness, they should be managed accordingly. There are, however, opportunities to advance data analyses methods. In addition, RWE should be viewed more critically since the data have not been assessed for quality.
Participants recommended the development of robust scientific methods for RWD to help make sure that relevant questions are asked and appropriate methods are applied to answer them and then advised to proceed with caution on the implementation of RWE.
A comment was raised that regulators and HTA organizations can develop methods to generate RWE, and a framework that outlines RWE use throughout the decision-making process. It would appear that there are numerous guidelines and best practice publications on RWE, but there are fewer applications of their incorporation in regulatory and HTA processes:
“We could develop a data analysis guidance that could help guide us when to make a recommendation with conditional evidence development to help make sure that we can reproduce the outcomes in the primary studies before we diffuse the health technology to a wider audience” (04 HTA agency).
Privacy Issues and Legal Aspects of RWD Collection
Organizations can be reluctant to share their data and information. Unless the privacy legislation in a jurisdiction permits health information custodians to share the patient's medical information without individual consent for specific reasons, agreements and consent from patients to collect their data would be necessary. Other participants mentioned that, in terms of confidentiality, security measures should be put in place, but there must be a balance between the individual right versus the common good.
Data access in some jurisdictions is also deemed a challenge due to the local privacy laws. To support informed healthcare decision making beyond the published literature, a minimum data set for hospitals can help increase the comparability of data collected on devices across hospitals.
Based on patient responses, they generally supported the application of RWE for decision making, and viewed it as an alternative data source for more treatment options. Although most patients would be willing to share their anonymized or de-identified medical data to researchers, healthcare providers, government and patient advocacy groups, they should always have the option to opt out at any time. On the other hand, some participants voiced concerns with the increased risk of discrimination if the patient's privacy and confidentiality is compromised, such as a reduction in their treatment options:
“From the patient's perspective, definitely clinicians, the researchers, and government are groups that patients are comfortable with having access to this data. They are trusted groups with the reasoning and with what they will be doing with this information and this data. The greater concern would be around groups like insurance companies and even the manufacturers and what they're going to be doing with this information and how it could benefit patients or hinder patients. Or, specifically to manufacturers, will they be using this information for a for-profit business?” (01 patient representative).
Options to Implement the Use of RWD and RWE
Mechanisms must be established for agreements and permissions to facilitate data and information sharing across jurisdictions. There seems to be mixed opinions, however, among participants on the feasibility and utility of national databases for medical devices. For instance, one participant stated that international jurisdictions have implemented them, while another participant found creating a national database or registry would be expensive as consent would be required from each jurisdiction:
“One of the challenges with registries is that they are not sustainable as they cost money and are time-consuming. They're usually too difficult and too expensive to maintain” (02 medical society).
Despite the resources required, a number of participants supported the use of registries to collect RWD on medical devices as they felt that registries had the capability to identify early and strong signals for new devices prior to market entry. In fact, one participant felt that registries can capture critical measurements in a consistent manner:
“Having a national registry would be of true value. There could be a role for Health Canada to speak to some of the provinces to convince them to release their data outside of the provinces, so we can have a national data set. It may go farther than if research groups try to do it” (05 clinician).
Participants felt that there are opportunities for partnerships with healthcare systems, healthcare providers, clinical societies, and industry to work together on this matter. In Canada, for example, a federal and provincial partnership is envisioned that incorporates some national strategies that are operationalized by the provinces. A participant commented that partnerships with organizations, such as the National Evaluation System for health Technology Coordinating Center (NESTcc), would be beneficial to leverage their resources to help understand the current gaps, study designs, and opportunities for future or long-term label expansions. NESTcc is a coordinating center based in the United States that provides services to organizations that seek to sponsor medical device/technology research based on high-quality RWD (11).
“On the reimbursement side, leveraging the HTA bodies that are thinking about real-world data is important. On the data side, we can also leverage [Medical Device Epidemiology Network] MDEpiNet and NESTcc on the data opportunities. On the academic side, there are a lot of interactions in health services research among those who use RWE in Canada and in the United States. For us not to leverage that is also a missed opportunity” (01 HTA agency).
In addition, international collaborations can increase the chances of harmonization across databases that collect RWD and access alternative data sources for therapeutic devices and laboratory tests that may not be available yet in a jurisdiction. A participant felt that some countries, including Canada, would benefit from industry conducting research in their country by shortening the amount of time required to set up a project.
Discussion
Thirty-four key informant interviews with forty Canadian and international stakeholders and patient representatives were conducted. Respondents indicated that applications for RWD and RWE include the PMS of medical devices, to inform reimbursement and investment decisions, and to measure and give feedback on medical device performance. RWD and RWE can also offer valuable insights across the product life cycle, including safety and use of medical device in “real-world” settings, and identify relevant study populations, comparators, and outcomes to be measured in prospective randomized controlled trials (RCTs). Furthermore, healthcare providers must be made aware of the uses of RWE for regulatory decisions and HTA recommendations to increase their motivation to collect quality data. Interdisciplinary and international collaborations were also deemed as paramount to gain an increased awareness of the perspectives and initiatives in this space. The results of the telephone interviews contributed to the key findings of our environmental scan on RWD and RWE for medical devices (5).
As per our study participants, partnerships and a willingness to share data across healthcare organizations and jurisdictions are necessary to help increase RWE uptake. RWE is a good example of where the clinical community, regulators, and HTA bodies can collaborate on the collection of data on a new health product in a healthcare setting (12). Facey et al. indicated that demonstration projects of highly innovative health technologies would facilitate discussions among stakeholders, including HTA producers, regulators, industry, clinician, and patients, to determine the type of RWE required, for what purposes, and details on the RWD collection for evidence-based decisions in the European Union (EU). They also suggested that participation in these demonstration projects can lead to the development of an EU multi-stakeholder learning network to enhance RWE use to inform decisions on highly innovative technologies (Reference Facey, Rannaheimo, Batchelor, Borchardt and de Cock13).
Similar to the challenges expressed in our telephone interviews, issues that can preclude a wider acceptance on the use of RWE in the EU from an HTA perspective include data access in several Member States, privacy concerns, and limited data sharing and availability (Reference Kanavos, Angelis and Drummond14). Opportunities to overcome some of these challenges related to the use of RWD and RWE in the HTA context were discussed at length at the 2019 HTA International Global Policy Forum. Recommended actions to improve the quality and acceptability of RWE include an understanding on the types of questions that RWE can help to address, an increased transparency in the methods and standards employed for RWD collection and RWE generation, the development of common data models, and better integration of data collection in routine clinical practice (Reference Oortwijn, Sampietro-Colom and Trowman15).
In collaboration with our HTA partners, Health Canada will employ the results of the environmental scan (4), that includes the telephone interviews, to develop a framework to optimize the use of RWE for medical devices, including regulatory pre-market and post-market decision making (3;5). Results of the environmental scan also helped to guide the development of a joint strategy with the pan-Canadian HTA Collaborative (PCC) to enhance the use of RWE for medical devices (5). This strategy outlines the main activities and outputs for the optimization of RWE use to support regulatory decisions and HTA recommendations. As per the document, Health Canada and the PCC will prepare a framework that will direct the development of guidance documents on the processes and methods on the use of RWE for medical devices. This activity includes a number of tasks, such as a survey that was conducted to identify how RWE is used at Health Canada and among the PCC members. The results will serve as input in the RWE framework development and will help identify evidence gaps for making regulatory decisions and HTA recommendations (16).
Limitations
Our study is not without its limitations. As telephone interviews were conducted for practical reasons, the interviewer was unable to see the participant's physical gestures or nuances. Instead, she relied on their audio cues. Purposive sampling was applied, so participants were not randomly selected. They were invited to participate to provide diverse perspectives according to research interests, specialty areas, and industry. Perspectives from participants residing in several jurisdictions, such as British Columbia, Saskatchewan, the Atlantic provinces, except Nova Scotia, and the Territories, were not part of the interview responses. There was a potential for responder bias, where individuals, who were more interested in RWD and RWE for medical devices compared with their colleagues, participated in the interviews.
Conclusions
Our study presents the findings from our thirty-four telephone interviews of forty key informants on RWE for medical devices. Key concepts derived from the interviews centered on the current and future RWE use for medical devices, considerations for the organizational, medical, scientific, and legal aspects and privacy issues of RWD collection and RWE generation, and options to implement the use of RWD and RWE. Although RWD and RWE can add value in safety and effectiveness assessment on the use of medical devices in routine practice, partnerships and a willingness to share data are necessary to increase RWE uptake. These interviews were one component of an environmental scan on the use of RWD and RWE for medical devices in Canada. As outlined in the joint RWE strategy between Health Canada and the PCC, results of the scan will be used to inform the development of an RWE framework to support regulatory decisions and HTA recommendations for medical devices.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462320000859.
Acknowledgments
The authors would like to acknowledge that some questions in the key informant interviews were obtained with permission from Dr. Yvonne Bombard. They are based on her research with CanREValue on the incorporation and use of RWE for cancer drugs and were adapted to meet the scope of our project. Also, the authors are grateful for Vasantha Narasimhan's review of and input to the qualitative analysis of the interview responses. Finally, the authors would like to acknowledge Nicole de Guia, Dr. Christopher McCabe, Michael Paterson, and Dr. Harindra Wijeysundera for reviewing the environmental scan, “RWD and RWE for Medical Devices in Canada.”
Funding
None.
