Although patient safety is a priority within healthcare across the globe, medical errors still cause a considerable number of deaths. Factors contributing to medical errors are rapid changes in healthcare systems, increased use of medical devices, the quickening pace of work and the increased complexity of medical devices (Reference Balka, Doyle-Waters, Lecznarowicz and FitzGerald1). For example, in the United States, there are an estimated 454,383 adverse device-related events per year (Reference Hefflin, Gross and Schroeder2). The more complex a medical device is, the more difficult it is to recognize and control the hazards associated with its use (Reference Wagner3).
There is a great variety between different medical devices and their associated risks. For example, bandages are low risk for the patient, while medical ventilators are high risk for the patient. The risk associated with a medical device is a combination of the hazards, the probability and the consequences of potential adverse events. Different risks can be evaluated to determine their acceptability. If a risk is judged as too high, adequate measures for risk reduction should be implemented (Reference Gullikson, David and Blair4). One of these possible measures is to train the users of the associated medical device (Reference Tawfik, Ouda and Abd El Samad5;Reference Karsh6). However, training of staff is resource-intensive (Reference Gullikson, David and Blair4). The amount and type of training deemed adequate depends, among other things, on the risk of the medical device (Reference Hamilton7). The risk of a medical device can be evaluated using a risk assessment tool developed for this purpose (Reference Tawfik, Ouda and Abd El Samad5).
There are several assessment tools available to evaluate the risk of individual medical devices. The risk can be assessed at a national level, which is often the case before a medical device enters the market. The classification tools used to evaluate these risks vary from country to country (8). In addition to those at a national level, there are risk assessment tools specifically developed for use within hospitals. An example of such a tool has been developed by the American Society for Healthcare Engineering (9). This risk classification tool is based on: function of the device, risk for the patient, and maintenance of the device (Reference Collins10). Other criteria on which risk assessment tools can be based are: the degree of the invasiveness, the severity of adverse events, and the body system affected (8).
Although it is unknown whether used risk assessment tools are adequate, it is likely that hospitals use the risk assessment tools to ensure and improve patient safety. For example, hospitals could use the risk associated with a certain medical device to determine the amount and type of training. As it is unknown if and how risk assessment tools are used to determine the amount and type of training, this paper aims to explore this subject in Dutch hospitals. The goal of this study is to explore the risk assessment tools and criteria used to assess the risk of medical devices in hospitals, and to explore the link between the risk of a medical device and how those risks impact or alter the training of staff.
METHODS
A questionnaire was sent to all Dutch hospitals to collect quantitative data about the implementation of a national guideline (Reference Porte, Meijs and Verweij11). This guideline was developed to support the risk management and safe application of medical devices in hospitals. The guideline facilitates the safe implementation, use, and disposal of medical devices by providing support and interpretation regarding risk management and safe use of medical devices in patient care.
Development of the Questionnaire
The questionnaire was developed based on literature and expert opinions, then discussed with researchers in the research group. To achieve face and content validity, the questionnaire was piloted in the taskforce Medical Devices of the Dutch Hospital Association. The Dutch Ministry of Health, Welfare and Sport, the Dutch Hospital Association, and the Netherlands Federation of University Medical Centers were asked to comment on the questionnaire. The final questionnaire contained thirty-seven questions, of which three were focused on risk assessment tools and their link with training (Table 1). Hospitals were asked to send documentation about their risk assessment tools and the training systems developed based on the risk assessment.
Questions on Risk Assessment Tools in the Questionnaire Sent to All Dutch Hospitals
FDA, Food and Drug Administration; MDD, Medical Device Directive.
Participants
The paper-based questionnaire was sent to all eighty Dutch hospitals, both university and general, in December 2015. The list of hospitals was obtained from the Dutch National Atlas of Public Health 2014 (12). In the case of hospitals with multiple locations, the main site was selected. When there was no clear main location, the questionnaire was sent to all. Questionnaires were addressed to the board of directors. The cover letter requested that the questionnaire be delivered to the person responsible for implementation of the national guideline in their hospital, as well as explaining the goal of the research and stating that results were to be treated confidentially. Three and, if necessary, 4 weeks after the first questionnaire was sent, two reminders were sent to the nonresponsive hospitals. Following these reminders, an e-mail was sent to the board of directors of the hospitals that had not responded. In this e-mail, permission was asked to contact the person responsible for the implementation of the national guidelines by telephone. Finally, all hospitals that did not respond were telephoned to request their participation in the study.
Analysis
Following manual entry of the data, 10 percent of the data were checked for accuracy. An error rate of less than 1 percent was considered acceptable. The responses to the questionnaires were analyzed using descriptive statistics in Stata 14 (StataCorp, 2015).
RESULTS
Of the eighty hospitals, sixty-five returned the questionnaire (81 percent response rate). Nine hospitals were not able to participate in the study due to time constraints, and three did not provide a reason for nonparticipation. The responsible employees in the remaining three hospitals could not be reached. Approximately one-third of the respondents (26 of 62) were medical physicists, while nine were the head of their hospitals’ medical devices department. In total, thirty-three hospitals provided additional documentation regarding their risk assessment tools or training systems. Eight hospitals sent their risk assessment tools, six sent their training systems and six sent both. The remaining thirteen hospitals provided additional information on the questionnaire.
Risk Assessment Tools
Every hospital that returned the questionnaire uses a risk assessment tool. The majority of hospitals use a tool developed internally (40 percent), while the next most common tools in use were developed by the American Society of Healthcare Engineering (ASHE) (19 percent) (9), the Medical Device Directive (11 percent) (13) and the US Food and Drug Administration (5 percent) (14). Furthermore, there were sixteen hospitals (25 percent) that use another tool. Tools developed by hospitals or other tools often use the ASHE tool as a base for their risk assessment method. The tools are based on different criteria designed to assess the risk of a medical device. Commonly used criteria for assessing the risk of a medical device are: the function of the device (92 percent), the severity of adverse events (88 percent) and the frequency of use (77 percent). The different criteria and the percentage of risk assessment tools in which the different criteria are applied can be seen in Figure 1. In most assessment tools, the medical device is awarded points for every criterion, with the total number of points determining the risk category of the device. The number of criteria used to assess a tool ranges from one to nine. Criteria differ between tools, but also among the same tools used in different hospitals. Two examples of risk classification criteria, function of the device and severity of adverse events, can be found in Table 2.
Percentage of risk assessment tools in which the different criteria are used (n = 62).
Examples of Criteria for the Risk Classification of Medical Devices
Training and Examination
Fifty-six hospitals responded to the question as to whether they organize their training systems based on the risk assessment of their medical devices. Of these hospitals, forty-seven base their training on the risk of medical devices, while nine hospitals do not. Some training systems are not solely based on the risk of medical devices, but also on the frequency of use.
Hospitals provided additional information about their training system and the type of training, but not all hospitals were able to provide information on all risk levels. twenty-six hospitals provided additional information on training for low- and high-risk devices, while twenty-five hospitals did so for medium-risk devices. The additional information was collected from the open field in the questionnaire or from the documentation added by hospitals. For a certain level of risk, it was possible for a hospital to require multiple types of training. For high- (n = 18) and medium-risk devices (n = 17), most hospitals provide training followed by education. In this context, “education” is defined as information-delivery education focused on theoretical knowledge, while “training” is defined as practical training focused on the skills of the user. For low-risk devices, most hospitals only require staff read the relevant user manual (n = 15). The training provided for the different risk levels can be seen in Figure 2.
Type of training for healthcare staff based on the risk of a medical device. The categories (high-risk [n = 26], medium-risk [n = 25], and low-risk [n = 26]) were answered by a different number of hospitals. It was possible for hospitals to provide more than one answer for each category.
Some hospitals provided additional information about staff examinations based on the risk assessment. Of the hospitals that provided information about low-risk devices (n = 15), none conducted examinations for staff. For the medium-risk devices, thirteen hospitals provided additional information, with seven requiring no examination, three requiring a self-assessment and three holding an examination. Of the twenty-one hospitals that provided information about examinations for high-risk devices, all conduct examinations (n = 19) or self-assessment (n = 2).
The frequency of training and/or examinations is dependent on the level of risk of the device. Several hospitals provided additional information on the frequency of training and examinations. For low-risk devices, several hospitals (8/19) train their staff only once. This training occurs when the new device is introduced or when staff start working at the hospital. For medium-risk devices, the biggest group of hospitals train and/or examine staff once every 3 years (6/16), while two do this more often. For high-risk devices, the biggest group of hospitals train and/or examine staff once every 3 years (8/19), while five do so more frequently.
DISCUSSION
In general, Dutch hospitals base their training for medical devices on the risk of the medical device. With increased risk, the amount and type of training and examination also increases. This was found to be independent of the type of risk assessment tool used. The use of a risk assessment tool and a plan for the training of medical device use are mandatory for Dutch hospitals. However, it is not mandatory for the training to be linked to the risk associated with a medical device. The finding that most hospitals base their training on the risk of a medical device could indicate that risk assessment is a helpful tool in determining training needs.
Risk assessment tools used by hospitals consider several criteria. The most common criteria are: the function of the device, the severity of adverse events, and the frequency of use. These criteria are more extensive than those used by the European Union (EU) and United States. Risk assessment in the United States is mainly based on the potential harm for the patient (Reference Guez and Garg15), while in the EU it is focused on the invasiveness of the device (Reference French-Mowat and Burnett16). In the majority of tools, the invasiveness of the device is not considered as a criterion, although the potential harm for the patient is. Although hospitals use different tools, it is unknown whether this can lead to variances in classification for the same medical device. Such variances could lead to a difference in the perceived risk and might influence patient safety if the assessed risk is too low (Reference Hwang, Sokolov, Franklin and Kesselheim17).
Alongside the criteria, the number of variables considered could influence the risk classification (Reference Wyatt and Altman18;Reference Harrell, Lee, Matchar and Reichert19). In Dutch hospitals, tools vary in number of criteria, from one to nine. When more criteria are considered, more potential influencers of risk are assessed. The drawback of considering too many criteria is that the most important will have less influence (Reference Oliver, Daly, Martin and McMurdo20;Reference Healey, Osler and Rogers21). A possible solution, which is already in use, is to prioritize and weigh the different criteria against each other (Reference Harrell, Lee, Matchar and Reichert19;Reference Jennett22). Further research could help to identify the key criteria, as well as the optimal number of criteria. To develop a tool, the multiple steps of risk assessment could be applied, namely: identification of the risks, assessment of the risks and identification of a solution (23). Historical data are often used in safety research to identify possible improvements (Reference Glendon, Clarke and McKenna24). In the case of medical devices, it is possible to use past incidents and errors to identify those that are high-risk. However, this method will not predict the risks of future medical devices. A more evidence-based risk assessment tool should be developed for hospitals, where the necessary evidence could be collected through analysis of the different criteria and the degree of influence they have on risk assessment and safety.
Furthermore, criteria could be interpreted differently and, therefore, influence the assessed risk of a device. For example, frequency of use is considered in 77 percent of the risk assessment tools in Dutch hospitals, and many hospitals stated that the risk associated with a device increases with more use. In contrast, some hospitals stated that a device is risker when used less often. Therefore, both frequent and infrequent use of a medical device could increase its risk. Frequent use of a device increases the chance that adverse events occur. Moreover, users could become unaware of the risks (25). However, infrequent use could increase the risk of adverse events as users do not have a thorough understanding of the instructions for device operation (26), or because they are unaware of the risks (25). When a risk assessment tool is developed, it is not only the criteria that should be evaluated, but also the way criteria are applied.
The results of the questionnaire suggest that the risk of a medical device guides hospitals to determine which methods of training and examination are most suitable to improve safe use. However, the quality and results of training were not assessed as part of this research. To improve safety and decrease the risk of working with medical devices, it is necessary to use validated training programs (Reference Greiner and Knebel27). Moreover, training should be tailored for staff, who may work with the medical devices in diverse ways. These differences were not found in the policies sent by Dutch hospitals, however, it could be that these differences are present in practice. Further research should reveal whether differences in training occur. Moreover, research should examine the clinical effects and safety of risk assessment tools, as well as the combination of risk of a medical device and the training provided for its use.
To the knowledge of the authors, this research is the first to provide an overview of the risk assessment tools used in Dutch hospitals and their link with training and examination. For patient safety, it is recommended that hospitals use the optimum tool to assess the risk of medical devices. However, the most effective method of risk assessment and the link between risk classification and training is not known. Moreover, the differences between hospitals indicate that it is difficult to identify the optimum tool. Lack of knowledge as to which is the optimum tool can lead to difficulties in risk assessment and subsequent training (Reference Flottorp, Oxman and Krause28). Hospitals should receive support to determine which tool is best to use. A starting point could be to assess the number of incidents with a certain medical device, and the changes in number of incidents when risk assessment tools were implemented or altered. Furthermore, the assessed risk of medical devices could be compared with the perceived risk by staff and the amount of training they desire.
One strength of this study was the response rate of the hospitals. All Dutch hospitals were asked to complete the questionnaire and 81 percent did so. Of the hospitals that returned the questionnaire, fifty-six of the sixty-five (86 percent) completed the question about the link between risk classification and training, although only twenty-nine hospitals provided additional information on this subject. This might be because the question was not clear enough: It was an open-ended question that asked if there was a link, and what the link was, between the risk of a medical device and the training. When the questionnaire was developed, the researchers did not have base information that would enable the asking of more specific questions, for example, about registration of training. By suggesting that attachments could be added and using an open-ended question, the researchers received information from hospitals on a broad range of subjects. This provided an abundance of additional information about how training and examination take place.
A limitation of the study was that the questionnaire was completed by staff who were involved with the implementation of risk assessment tools, leading to a risk of socially desirable answers. However, this is not expected to be the case due to the nature of the questions and the risk assessment and training policies that were sent. Moreover, the questionnaire was sent to Dutch hospitals and, therefore, only provides insight on the situation in the Netherlands. However, the results indicate which risk assessment tools are used in hospitals and in which way training can be linked to the risk of a medical device. This knowledge can be applied in hospitals outside the Netherlands.
In conclusion, all Dutch hospitals use tools to assess risks associated with the use of medical devices. There is wide variety in the form and content of these tools. The most common criteria in risk assessment tools were found to be: the function of the device, the severity of adverse events, and the frequency of use. The different tools could lead to varying classifications for the same medical device. The risk associated with a medical device is often used to determine the amount and type of training and examination. When the risk of a medical device is higher, the training and examination are more intensive and compulsory. Understanding the link between the risk of a device and the amount and type of training could improve the proficiency of users and, therefore, might influence patient safety.
CONFLICTS OF INTEREST
The authors have nothing to disclose.
