Introduction
Continuously improving quality of care by applying the latest scientific insights is considered highly relevant nowadays but has been shown to be very difficult to achieve in long-term care (e.g. Appelhof et al., Reference Appelhof2018; Leontjevas et al., Reference Leontjevas, Gerritsen, Koopmans, Smalbrugge and Vernooij-Dassen2012; Sharkey et al., Reference Sharkey, Hudak, Horn, Barrett, Spector and Limcangco2013; Zwijsen et al., Reference Zwijsen, Smalbrugge, Eefsting, Gerritsen, Hertogh and Pot2014a). For instance, the reported degree of implementation in several intervention studies was suboptimal, and a variety of barriers to the implementation of different interventions in nursing homes has been described (e.g. Appelhof et al., Reference Appelhof2018; Francke et al., Reference Francke, Smit, De Veer and Mistiaen2008; Mentes and Tripp-Reimer, Reference Mentes and Tripp-Reimer2002; Verkaik et al., Reference Verkaik, Francke, Van Meijel, Ouwerkerk, Ribbe and Bensing2011; Zwijsen et al., Reference Zwijsen, Smalbrugge, Eefsting, Gerritsen, Hertogh and Pot2014a).
In order to determine and optimize the actual contribution of an intervention to daily practice, the process of its implementation should be studied alongside the effects of the intervention (Hulscher et al., Reference Hulscher, Laurant and Grol2003). Moreover, the internal and external validity of an executed effect study may suffer from poor implementation (Leontjevas et al., Reference Leontjevas, Gerritsen, Koopmans, Smalbrugge and Vernooij-Dassen2012). If the quality of a study’s sample is insufficient (e.g. high sampling bias, insufficient sample size, excessive dropout rate), the validity of the study is compromised and its credibility and generalizability decreased. These features are commonly addressed when reporting effect studies. However, intervention quality, i.e. the extent to which the intervention is executed and the acceptance of the intervention by health professionals, is highly relevant as well (Leontjevas et al., Reference Leontjevas, Gerritsen, Koopmans, Smalbrugge and Vernooij-Dassen2012). Low intervention quality may hinder implementation of interventions in daily practice (Eldridge et al., Reference Eldridge, Ashby, Bennett, Wakelin and Feder2008; Glasgow et al., Reference Glasgow, Klesges, Dzewaltowski, Estabrooks and Vogt2006), which also holds true for poorly delivered and/or received implementation strategies, and encounter barriers to implementation (Leontjevas et al., Reference Leontjevas, Gerritsen, Koopmans, Smalbrugge and Vernooij-Dassen2012).
One relevant area for improving quality of care is psychotropic drug prescription among people with dementia with neuropsychiatric symptoms (NPS). Although a limited reduction in NPS has been reported, these drugs have considerable side effects (Seitz et al., Reference Seitz2013), and psychotropic drug treatment is still common (Janus et al., Reference Janus, Van Manen, Mj and Zuidema2016; Smeets et al., Reference Smeets2017), often inappropriately long (Gustafsson et al., Reference Gustafsson, Karlsson and Lovheim2013), and lacking proper indication (e.g. Lucas et al., Reference Lucas2014). Our PROPER study found, in fact, that only 10% of psychotropic drug prescriptions were fully appropriate according to guidelines (van der Spek et al., Reference Van Der Spek2016). Therefore, optimization of appropriateness of psychotropic drug prescriptions in long-term care is necessary. This is why we developed PROPER, a multidisciplinary medication review intervention (Smeets et al., Reference Smeets2013). This intervention was recently shown to be effective in increasing the overall appropriateness of psychotropic drug prescription (van der Spek et al., Reference Van Der Spek2018).
Nevertheless, before concluding on its contribution to daily practice and suggesting nationwide implementation of the PROPER intervention, the quality of the intervention and its implementation should be optimal. Therefore, the current paper aims to describe the process evaluation of the intervention’s quality and implementation process in the context of the PROPER intervention effect study.
Methods
Design and setting
The process evaluation was conducted alongside the implementation of the PROPER intervention. The PROPER effect study was a multicenter, cluster-randomized, controlled, pragmatic trial using parallel groups, with a duration of 18 months, and four biannual assessments (T0, T3). It was conducted from September 2012 to July 2014. The intervention (described below) was implemented in 13 dementia special-care units of 6 long-term care organizations. Another 16 units in 6 other organizations continued care as usual (see Smeets et al., Reference Smeets2013 and van der Spek et al., Reference Van Der Spek2018 for a detailed description). As reported in these papers, the PROPER study was conducted according to the principles of the declaration of Helsinki and the applicable rules in the Netherlands.
The framework for first- and second-order process evaluation of Leontjevas et al. was used for the process evaluation (Leontjevas et al., Reference Leontjevas, Gerritsen, Koopmans, Smalbrugge and Vernooij-Dassen2012). This framework is based on the framework of Steckler and Linnan (Reference Steckler and Linnan2002) and the proposed criteria for assessing internal and external validity (Eldridge et al., Reference Eldridge, Ashby, Bennett, Wakelin and Feder2008; Rothwell, Reference Rothwell2005). To investigate the internal and external validity of a study, the framework proposes collecting first-order process data to evaluate (a) the quality of the sampling, i.e. recruitment, randomization, and reach, which were already evaluated in the PROPER effect study (van der Spek et al., Reference Van Der Spek2018); and (b) the quality of the intervention, i.e. relevance and feasibility of, and satisfaction with, the intervention, as well as the extent to which the intervention was performed. Second-order data regard information on implementation, i.e. the implementation components delivered and received, and encountered barriers and facilitators.
Intervention components and stakeholders
In short, the PROPER intervention consisted of a structured and biannually repeated multidisciplinary medication review (Component 2) supported by organizational preparation and education (Component 1) and evaluation and guidance (Component 3). It was conducted by the responsible physician, pharmacist, and a licensed practical nurse (LPN); implementation was coordinated by a local implementation coordinator (IC). In the Netherlands, an elderly care physician is usually responsible for the treatment plan and employed by the long-term care organization (Koopmans et al., Reference Koopmans, Pellegrom and Van Der Geer2017). Pharmacists are permanently involved as consultants in Dutch nursing homes. In the Netherlands, LPNs work autonomously in providing care and are responsible for their individual actions and practice. They have responsibilities such as admitting new residents, monitoring resident care, supervising other nursing staff, performing ongoing assessments of residents’ physical and mental health, and explaining procedures and treatment protocols to residents. They administer medicine and injections, take vital signs, and provide basic bedside care (https://www.practicalnursing.org/lpn-jobs-description). LPNs have the opportunity to attend additional education in order to function as the responsible nurse and contact person of a small group of nursing home residents within a unit. These responsible LPNs (RLPNs) are the nursing staff members involved in our study.
For Component 1, at the start of the intervention, the physician, the pharmacist, the unit manager, and all nursing staff were invited to a joint information and training session provided by the Dutch Institute for Rational Use of Medicine (IVM). For the actual review (Component 2), two registration forms were developed. The PROPER preparation checklist was to be filled out prior to the review by an RLPN and discussed during the review. Using the checklist, they were to record resident observations for behavior and side effects and consult colleagues to complete the information. This was in addition to the preparation performed by physicians (e.g. checking medical history and laboratory results). The second form was the PROPER medication-change form, to be used to register the proposed medication changes during each review. Component 3 entailed stakeholder evaluation meetings guided by the IVM, to share experiences and points for improvement. Detailed information about the intervention is presented elsewhere (Smeets et al., Reference Smeets2013).
Each intervention unit had a small group of trained RLPNs (Component 1) to facilitate the presence of one of them at the reviews and evaluation meetings. In case of staff turnover, newly employed staff were briefed by the exiting stakeholder, the researchers, and/or the local IC, and received the training manual (Smeets et al., Reference Smeets2013).
Implementation strategies
Three strategies were applied to implement PROPER intervention: (a) appointment of a key stakeholder, (b) development of implementation materials; (c) establishing a help desk.
-
a. For each participating long-term care organization in the intervention group, an IC was assigned. The ICs’ tasks were being the primary contact person for all stakeholders and ensuring appropriate planning and organization of the intervention.
-
b. Together with the research team, the IVM developed an intervention manual containing implementation guidelines and instructions for drawing up an implementation plan. At the start of the intervention (Component 1), the IC and nursing home staff drew up the implementation plan together with the IVM, which was to be reevaluated during the evaluation meetings of Component 3.
-
c. As part of Component 3, a help desk was established for stakeholders to contact the IVM or the researchers with questions about the implementation.
Procedure and measurements
The intervention quality (i.e. the extent to which the intervention was performed, relevance and feasibility of, and satisfaction with, the intervention), the delivered and received implementation strategies, and barriers and facilitators to implementation were evaluated based on information from the ICs, physicians, pharmacists, and nursing staff involved in the PROPER effect study using four data sources (Table 1).
-
1. Web-based questionnaires were administered at all measurement points and contained predominant questions with a yes/no or 5-point likert response scale. For these questions, participants were asked to illustrate their answers. Furthermore, a number of open-ended questions were included. See below.
-
— Questions asked at T0: performed medication reviews, how often for each resident, who was present, how much time did they take per resident, and were guidelines used during the reviews. Open questions regarded how stakeholders prepared for the reviews, whether stakeholders were satisfied with the reviews, and what were possible improvement points.
-
— Questions asked at T1, T2, T3: the previous questions, plus, for intervention units: did the training affect resident care, if yes, how; was training knowledge used in medication reviews; were implementation materials used; how was communication experienced with the pharmacist, physician/nursing staff member, the IC, and the IVM; are there any other remarks about the training or reviews; what were barriers and facilitators (tick from a list and add others)?
-
-
2. Attendance lists, minutes, and evaluation forms of the different meetings. Evaluation forms consisted of closed questions to be rated from 0 (very bad) to 10 (excellent) combined with requests for elaboration. This included:
-
— Information sessions for involved physicians, RLPNs, and pharmacists on practical and organizational aspects of medication reviews.
-
— Training about psychotropic drugs for involved physicians, RLPNs, and pharmacists.
-
— Two evaluation meetings for involved physicians, pharmacists, and at least one of the involved RLPNs per unit.
-
-
3. Short, open telephone interviews with the physicians, nursing staff members, and/or IC after the medication reviews. Subjects included: who was present, the extent to which the materials were used, and experiences regarding the review.
-
4. Closing interviews with the ICs: at T3, the ICs were interviewed by one of the researchers. This was a semi-structured interview with questions about the intervention as a whole, expectations, and time investment.
Data sources used for the process evaluation
* physicians, pharmacists, and responsible LPN
** I = from intervention units; C = from control units
Analyses
Descriptive statistics of SPSS 20.0 were used to analyze the quantitative data. Barriers and facilitators were identified using content analysis (Elo and Kyngas, Reference Elo and Kyngas2008) of the interviews and web-based questionnaires. If more than one web-based questionnaire was available for a stakeholder, the most recent one was used. Exceptions are indicated in the text. The evaluation focused on the implementation in all participating units in general, but also on differences between these units.
Results
Participant characteristics
Twenty-one physicians (10 women) from intervention units were involved during the study. Nineteen provided information on their age in web-based questionnaires, which was 43.5 years on average (range 27–64) at their first measurement, and 17 reported their years of working experience, which was 10.1 on average (range 0–30). Fourteen physicians (11 women) from control units were involved during the study, with a mean age of 46 years (range 33–59). Years of working experience amounted to 11.4 on average (range 2–25).
The RLPNs involved were the staff members who participated in the collection of resident data for the effect study. For intervention units, this concerned 36 RLPNs (all women). Their mean age was 39.4 years (range 23–58) at their first measurement. Their experience as an RLPN was 5.9 years (range 0–24). Thirty-six RLPNs (all women) of control units were involved and provided data at baseline. Their mean age was 44.6 years (range 28–62). Their experience as an RLPN was 9.4 years on average (range 0–38).
For the intervention units, nine pharmacists (three women) were involved in the study. Their mean age was 42.4 years (range 34–61); working experience was 16 years on average (range 8–34).
Seven ICs were involved. Four were unit managers (one replacing another during the study), one a physician, one an overall care manager, and one a registered nurse. All were women, except the overall care manager.
Table 1 provides an overview of the data sources used in the process evaluation. The overall response rate for the questionnaires was 90%, with 88% for control units and 94% for intervention units.
Intervention quality
Extent of intervention performance
Combining the data sources, Table 2 shows the absence of the stakeholders during the execution of the three intervention components.
-
1. Preparation and education
With the exception of one pharmacist (Units 10 and 11), the physician, pharmacist, and at least one RLPN from every unit were present during the information session on practical and organizational aspects of medication reviews and the training about psychotropic drugs (95% participation rate).
-
2. Conduct of medication review
After the first review round, most participants mentioned having prepared the review in the questionnaires. However, when asked specifically about focusing on side effects in their observations—irrespective of the materials used—only 14 out of 30 RLPNs reported having done so.
Intervention materials used: the PROPER preparation checklist was used to prepare 56% of the reviews, decreasing from 85% of the units during the first review round to 39% during the third.
During the study’s required three rounds of medication reviews per unit, the physician, pharmacist, and RLPN were mostly present (95%). Table 2 shows further details. In general, the medication use of all participating residents from a unit was reviewed in one session. During the required three rounds, a medication review was conducted for each participating resident (100%), with an average duration of 13 minutes (range 9–20) per resident.
Intervention materials used: the PROPER medication change form was used in 64% of the sessions with no apparent differences between the review rounds. Alternatively, the physician only recorded the proposed changes in the medical file of the resident.
-
3. Evaluation and guidance
Participation rate at the evaluation meetings was 82%. Generally, an RLPN and physician were present. The pharmacist of the four units of two long-term-care organizations was never present (Units 10, 11, 12, 13). Table 2 shows further details.
Shortly after the start of the study, the researchers decided it was necessary to stimulate implementation by keeping track of the process more closely. Therefore, they attended 15 of the information and evaluation meetings to answer questions about the implementation and the study. Furthermore, the researchers intensified their support when considered necessary, for example when the IC could not perform all tasks (Units 10, 11) or if a previous period did not go according to plan (Units 3, 4, 12, and 13).
Absence of stakeholders at PROPER intervention components
Medication reviews in usual care
Notably, the web-based questionnaires showed that medication reviews had already been performed on 7 (out of 13) intervention units versus on 13 (out of 16) control units before the start of the trial, about once a year. Physicians from 1 of these 7 intervention units and 6 of the 13 control units prepared the reviews at that time. Before the trial, no RLPN was present for the medication reviews on the intervention units versus for half of the medication reviews in the control units.
During the trial, more than half of the 16 control group units performed medication reviews (T1: 9/16; T2: 10/16; T3: 9/16). An RLPN was present in 25%–50% of the reviews.
Relevance and feasibility
At baseline, all participating physicians and nursing staff members were asked, using an open question in the web-based questionnaire, where they could see room for improvement regarding the medication reviews in their usual care. Supplemental Table 1 shows that many of the mentioned points for improvement are taken into account by the PROPER intervention (see Table S1 published as supplementary material online attached to the electronic version of this paper). The most-mentioned improvement point for physicians was a wish for structural involvement of nursing staff; most mentioned by RLPNs was the wish to evaluate medication more frequently or more thoroughly with the physician.
Satisfaction with the intervention
As assessed through the questionnaires, physicians and RLPNs were positive about the quality of the structured reviews and, to a lesser degree, about the extent to which they could apply the knowledge they had acquired in the training during the medication reviews. The quality of the communication aspects was highly rated (Table 3).
Satisfaction with intervention among physicians and RLPNs
Note: Data collected from the last follow, up measurement (T1, T2, or T3) of every RLPN and physician.
1 Scale 1 (bad) to 5 (good)
2 Scale 1 (not at all) to 5 (completely)
The interviews (with ICs) and questionnaires (for physicians and RLPNs) show that the three intervention components (education, review, evaluation) were considered to be of great value, as was the involvement of nursing staff members. Other intervention characteristics were also positively rated: pharmacist involvement, nursing staff having a clear role, evaluation meetings, training, the education session, and intervention materials.
As for Component 1, attendees rated the training about psychotropic drugs 7.8 (range 5–10) (N = 70); the information session on practical and organizational aspects of medication reviews scored 7.4 on average (range 6–9; N = 60). The evaluation meetings (Component 3) both scored 7.8 (ranges 6–10 and 7–10; N = 35 and 38).
Consequences of the training and the subsequent medication review mentioned in the questionnaires were “more awareness of and knowledge about psychotropic drugs” and “an extensive and higher quality medication review.” RLPNs also mentioned “improved observation of the resident” and “a different attitude towards the resident.”
Intervention materials: physicians and RLPNs were satisfied with the materials (preparation form/medication change form/guidelines), as they were clear, useful, and applicable. The PROPER medication-change form was considered a duplication by some physicians. Most RLPNs reviewed the PROPER preparation checklist as a good, clear tool to prepare for the medication review, although some considered it redundant since they knew their residents well and found it unnecessary to record anything.
Points for improvement
Several stakeholders stated in the questionnaires and at the evaluation meetings that because the time periods between the reviews were rather long, they sometimes forgot what was expected of them. They said this resulted, for example, in insufficient preparation. Additionally, some physicians mentioned a remaining need for “better organization, preparation and follow-up,” and RLPNs mentioned “more awareness and being more analytical” and “evaluating medication more quickly and more often.”
Feasibility for and the contribution of the nursing staff may have been suboptimal. Several RLPNs considered the training, which was medication focused, very difficult. Furthermore, although communication with the physician and pharmacist during the intervention period was generally highly rated, the conversation between these stakeholders during the medication reviews was often too pharmacologically oriented for the RLPN present.
Implementation
Execution of implementation strategies
-
1. Key stakeholder: most ICs were present during the entire intervention period. In one organization, the IC (a unit manager) left the unit during the intervention. The replacing unit manager and the researcher stepped in to take over.
-
2. Implementation materials: all ICs used the IC manual during the intervention. The implementation plan was drawn up at the start of the intervention for all units by ICs, nursing home staff, and IVM and was reevaluated and adapted during the evaluation meetings.
-
3. As for the help desk, the IVM and the researchers were rarely contacted via telephone or email with questions about the intervention. Most questions regarded the measurements for the effect study. However, the intensified interaction—especially during evaluation meetings and the telephone interviews after the medication reviews—provided often-used opportunities to ask questions.
Barriers and facilitators to implementation
The barriers and facilitators that were identified from the different data sources (see Table 1) can be categorized into four themes: time, organization and planning, staff turnover, and attitude and communication (Table 4).
Barriers and facilitators mentioned in the different data sources
Notes: IC = intervention coordinator.
Gray cells represent barriers; white cells represent facilitators.
Data sources: 1interview with the IC; 2 minutes and 3evaluation forms of the different meetings; 4 web-based questionnaires.
Time
Time was mentioned as a barrier, mostly in terms of the extra time required to perform the measurements for the effect study or the intervention, and sometimes in terms of time constraints on the unit. Notably, the time investment necessary to participate in the study measurements was a barrier to implementing the intervention. Furthermore, the PROPER intervention review took more time than the usual medication reviews because of the interdisciplinary nature and the required follow-up. Although medication is often changed in a unit, the entire unit was reviewed in one go, resulting in many medication changes for numerous residents simultaneously. Lack of time was mentioned as the reason for ceasing to fill in some of the intervention forms. Some of the staff involved also mentioned a lack of time to perform an in-depth review. Support of higher management by financing extra time was said to facilitate the implementation.
Implementation coordinator:
The study took a lot of time and was imposed by higher management, who realized only later on what participating really meant. Hours were made available, but especially for nursing staff members who work full-time, compensating was not possible, which means loss of hours intended for the care of residents.
Physician:
I really liked participating in the study. But it is very time-consuming and sometimes difficult to find the time, especially the extensive web-based questionnaire.
Organization and planning
The IC was key to ensuring that the execution of the intervention went according to plan. If the IC was less involved, implementation fell behind, and the researchers had to take on the role of IC (Units 10, 11). Involved staff mentioned that the intervention became a burden when coordination failed. Due to poor planning, the pharmacist was not present at some of the reviews. The process feedback that was provided at the evaluation meetings was considered a facilitator. Furthermore, the researchers noticed that the intervention was easier to implement in the long-term-care organizations where a medication review was already part of care prior to the study.
Implementation coordinator:
My role as implementation coordinator was certainly necessary; coordination was needed to enable other stakeholders to focus on their tasks, such as completing questionnaires and conducting medication reviews.
Physician:
The role of the implementation coordinator is really significant and necessary for effective implementation.
Staff turnover
Due to staff turnover, several stakeholders (RLPN, pharmacist, physician) were replaced during the study (Table 5).
Turnover of stakeholders on intervention units (N = 13)
* In several cases, a stakeholder returned or started working at another participating unit
As for participation in the intervention, Table 5 shows that there was only one unit without stakeholder changes during the implementation (Unit 5). The main reasons for staff turnover in both groups were transfer to other units or job switches. The RLPNs also changed because of maternity leave or no longer being the responsible contact person for a participating resident. Turnover of physicians was a specific barrier because the training could not be repeated. Some newly involved stakeholders reported not knowing what was expected of them and did not properly prepare the review.
Implementation coordinator:
There have been a lot of changes in physicians, and it was difficult to get everybody up to date with the study and the procedure.
RLPN:
It would help if we had the same physician from now on.
Physician:
The medication reviews are very difficult if you are newly employed and did not attend the training. It is a pity that the training is not repeated.
Attitude toward the intervention
The attitude of those involved was important for the success of the intervention. Many physicians, nursing staff members, and unit managers/ICs complemented and reinforced each other in their enthusiasm and cooperation, for example during the evaluation meetings. Stakeholders reported that communication during the reviews was enjoyable. Despite the time investment, most of the staff remained enthusiastic, saw the added value of the intervention, and wanted to contribute to its success, as reported in the web-based questionnaires. This is probably also due to the timing of the study; long-term care organizations had already been in the process of optimizing and reducing psychotropic drug use.
Implementation coordinator:
Everyone was intrinsically motivated and we were already trying to use as little medication as possible, and this study was in line with this transformation.
RLPN:
The intervention is a good way to be critical and alert together with the physician and pharmacist and implement an appropriate medication plan by considering observations and changes in health care and well-being of the resident.
RLPN:
It requires some time investment, but you also get good results. So it is worth the investment.
Discussion and implications
This process evaluation showed that the quality of the PROPER intervention during the effect study was generally sufficient with regard to the participation rate in education sessions, medication reviews, and evaluation meetings, although this varied between units. Two units (10,11) had particularly lower participation rates, which was predominantly caused by absence of the pharmacists and a less-involved implementation coordinator. The intervention’s relevance and feasibility were considered high. However, the education component and the communication between physician and pharmacist during the medication reviews was not sufficiently comprehensible for all RLPNs. The implementation strategies were rated positively, although the help desk was rarely used. Alternatively, the researchers increased involvement in supporting the IC in some units and increased interaction with stakeholders. Several barriers to implementation were identified, regarding the required time investment, planning and organization, and frequent staff turnover leading to discontinuity. The stakeholders considered having an implementation coordinator to guard the planning highly facilitating. Additionally, the positive attitude toward the intervention, including the support of higher management, was an important facilitator.
The local implementation coordinator was important for the success of the implementation, as was the case in several other studies into long-term care (Leontjevas et al., Reference Leontjevas, Gerritsen, Koopmans, Smalbrugge and Vernooij-Dassen2012; Sharkey et al., Reference Sharkey, Hudak, Horn, Barrett, Spector and Limcangco2013; Zwijsen et al., Reference Zwijsen, Smalbrugge, Eefsting, Gerritsen, Hertogh and Pot2014a). Additionally, the positive attitude toward the intervention was an important facilitator. In general, the readiness to change in nursing homes is not very high (van Beek and Gerritsen, Reference Van Beek and Gerritsen2010). However, this appeared not to be a problem for the PROPER intervention. Compared with other recent intervention studies in the Netherlands (Appelhof et al., Reference Appelhof2018; Leontjevas et al., Reference Leontjevas, Gerritsen, Smalbrugge, Teerenstra, Vernooij-Dassen and Koopmans2013; Zwijsen et al., Reference Zwijsen2014b), the extent of performance was higher. Comparing the interventions, the PROPER intervention appears less complex and closer to usual care, involving fewer professionals. Also, the time investment is highly clustered toward specific moments, making it easier to manage. Importantly, the subject was highly topical; all care organizations in the Netherlands are pressed to lower their inappropriate psychotropic drug use in people with dementia.
Regarding points of concern for the implementation, turnover and understaffing of nursing staff were important organizational barriers. These barriers are commonly reported (Resnick, Reference Resnick2013) and are difficult to influence. Time is also a commonly reported barrier that was present in our study (e.g. Appelhof et al., Reference Appelhof2018; Zwijsen et al., Reference Zwijsen, Smalbrugge, Eefsting, Gerritsen, Hertogh and Pot2014a). Interestingly, Rosemond et al. (Reference Rosemond, Hanson, Ennett, Schenck and Weiner2012) did not find time to be relevant, nor lack of financial investments or staff training, which are often reported barriers as well (Resnick, Reference Resnick2013). Alternatively, they suggest that the three most important criteria for successful implementation might be confidence of nursing teams in their ability to meet the change goals, their belief that change will contribute to improvement in daily routines, and commitment of management. Although the latter was found to be present in our study, the other two were not measured but may have contributed to our positive implementation results.
Compared to other papers on implementation and process evaluations in dementia care that were recently published in International Psychogeriatrics, several of the facilitators that were found were similar, for instance having a dedicated IC (Hendriks et al., Reference Hendriks, Meiland, Gerritsen and Droes2018; Van Mierlo et al., Reference Van Mierlo2018), a committed management (Buist et al., Reference Buist, Verbeek, De Boer and de Bruin2018), and available time (Tropea et al., Reference Tropea, Logiudice, Liew, Roberts and Brand2017; Van Mierlo et al., Reference Van Mierlo2018). Our study specifically indicates the importance of a positive attitude of all stakeholders toward an intervention—including organizational management—and, notably, the quality of their communication for success of implementation. Perhaps the most valuable addition of the current paper is that it not only focuses on barriers and facilitators (Buist et al., Reference Buist, Verbeek, De Boer and de Bruin2018; Loi et al., Reference Loi, Mazur, Huppert, Hoy, Swan and Lautenschlager2017; Tropea et al., Reference Tropea, Logiudice, Liew, Roberts and Brand2017; Van Mierlo et al., Reference Van Mierlo2018)—in one paper complemented with impact of the intervention (Hendriks et al., Reference Hendriks, Meiland, Gerritsen and Droes2018)—but that it describes a comprehensive process evaluation resulting in information on aspects that may have influenced the effects of the intervention and can now be included in effect analyses. Although we have already published about the effects of the intervention and found that it positively influenced appropriateness of psychotropic drug use (van der Spek et al., Reference Van Der Spek2018), the results of this process evaluation call for additional sensitivity analyses. First, the variation in quality of the implementation that was found between the intervention units could be taken into account. Second, the process evaluation revealed that the medication reviews in control units before baseline were of higher quality and often also included a member of nursing staff, which may have decreased the effects found in the performed intention-to-treat analyses. Third, turnover, which may have reduced the quality of implementation, was frequent and varied between units, implying that accounting for the turnover rate of units may be advisable.
A first limitation of this study may have been the timing of evaluation through the web-based questionnaires and meetings. In the study’s setup, the evaluation was performed several months after the medication reviews, possibly resulting in lower quality responses. Second, Table 4 shows that the LPRNs did not mention barriers and facilitators in multidisciplinary group evaluations, but used questionnaires. This is a phenomenon more often observed among nurses (Liberati et al., Reference Liberati, Gorli and Scaratti2016; Tjia et al., Reference Tjia, Mazor, Field, Meterko, Spenard and Gurwitz2009), which should receive attention in future evaluations of multidisciplinary care. Third, although the PROPER intervention incorporated most of the mentioned improvement points of usual care, the intervention was not adapted locally depending on the mentioned barriers. Recent research shows that implementation may be facilitated not only by adapting the implementation process but also the intervention itself (Day et al., Reference Day, Kenealy and Sheridan2016).
Although we have already published about the effects of the intervention and found that it positively influenced appropriateness of psychotropic drug use (van der Spek et al., Reference Van Der Spek2018), the results of this process evaluation call for additional sensitivity analyses. First, the variation in quality of the implementation that was found between the intervention units could be taken into account. Second, the process evaluation revealed that the medication reviews in control units before baseline were of higher quality and often also included a member of nursing staff, which may have decreased the effects found in the performed intention-to-treat analyses. Third, turnover, which may have reduced the quality of implementation, was frequent and varied between units, implying that accounting for the turnover rate of units may be advisable.
Some changes to the intervention need to be made to make it more nursing staff friendly: adjustments to the educational session to the predominantly pharmacological orientation of the communication during the reviews can both be addressed in the education component. Education could also include the importance of structured preparation, observation, and registration for medication reviews. The identified discontinuity due to turnover (including of physicians) might also be addressed by developing an e-learning module to provide continuous education that incorporates communication instructions. Digitizing the intervention may also have important benefits. Given that less than half of the RLPNs focused on side effects in their preparatory observations, a digital support system, including digitized preparation, could force this, thus increasing quality. Also, the information on what is expected of participants needs to be highly accessible, given their statements that sometimes they forgot what was expected of them. Digitization could be of help here as well.
Although performing medication reviews based on the status of individual residents might be best, organizing a multidisciplinary review more often than biannually will probably be impossible in practice. Additionally, to further increase implementation and sustain intervention, the local implementation coordinator could be even more intensively involved and monitor the continued use of the intervention after conclusion of the implementation trajectory. Last but not least, more attention should be given to two other crucial stakeholders: residents and their relatives. Their involvement is necessary for successful and tailored application of psychotropic drugs. Changes in medication as a result of the intervention were commonly discussed with the residents’ relatives, but how their involvement has taken shape was not recorded. Therefore, in future evaluation of implementation, these stakeholders should be included. Implementation strategies, such as information materials and incorporating meetings with residents and their relatives in the implementation process, may also increase their involvement.
In conclusion, the implementation of the PROPER intervention was successful. PROPER appeared to be feasible, and its implementation requires extra time and commitment of stakeholders, a committed implementation coordinator, and a strategy to accommodate staff turnover. Some arguments for sensitivity analyses emerged. The Dutch Health Care Inspectorate advises an annual medication review. The results of our process evaluation suggest that structuring this procedure and having a member of the nursing staff present are of added value.
Description of authors’ roles
Debby Gerritsen co-designed the study and supervised the data collection. She wrote the paper together with Erica de Vries, who also performed the data analyses and performed part of the data collection. Klaas van der Spek and Claudia Smeets developed the questionnaires, collected the data, and assisted in writing the paper. Martin Smalbrugge, Syste Zuidema, and Raymond Koopmans co-designed the study, data collection, and the data analysis, and assisted in writing the paper.
Funding
This work was funded by the Netherlands Organisation for Health Research and Development (ZonMw) under grant number 113101005. The sponsor had no role in the conception, design, analysis, or interpretation of data or in the drafting, review, or approval of the manuscript.
Acknowledgments
This study is a collaboration between the sections for elderly care medicine of three Dutch University Medical Centres and the Dutch Institute for Rational Use of Medicine and is supported by the collaborative organization of long-term care (ActiZ) and the Dutch Health Care Inspectorate. Netherlands Trial Register (NTR3569).
Conflict of interest
The authors report no conflict of interest.
Supplementary material
To view supplementary material for this paper, please visit https://doi.org/10.1017/S1041610219000577
