Policy makers, healthcare managers, and health technology assessment (HTA) producers are increasingly interested in exploring strategies for incorporating users’ perspectives in HTA activities (Reference Bridges and Jones1–Reference Lehoux and Williams-Jones4). Considering the opinion and experiential knowledge of health-care service users could result in a more comprehensive assessment of the value of health technologies and of their impact on the health of populations (Reference Bridges and Jones1–Reference Facey, Boivin and Gracia3). The literature reports on a few experiences of user involvement in local HTA, but best practices regarding who should be involved, as well as when and how, remain unknown (Reference Gagnon, Desmartis and Lepage-Savary5).
FRAMEWORK FOR USER INVOLVEMENT APPLIED TO A SPECIFIC HTA: THE ALTERNATIVES TO RESTRAINT AND SECLUSION
To inform decision making and practice in this domain, we developed a reference framework for user involvement in HTA at the local level (see Table 1). This framework is based on a systematic review of the literature on experiences of patient participation in HTA (Reference Gagnon, Desmartis and Lepage-Savary5) and on interviews conducted with HTA stakeholders in the Province of Quebec (Canada) (Reference Gagnon, Desmartis and Gagnon6). It is also inspired by other conceptual frameworks for patient and public involvement in HTA (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis7–Reference Tritter10).
Framework of Patient Involvement in HTA at the Local Level
a Types of participants: “Specialists”: patients (including their relatives and representatives) directly affected by the technology that is being evaluated; “Generalists”: patients who represent all current or potential service users, who may be represented, for example, by members of the Users' Committee.
The framework (Table 1) illustrates different modalities of user involvement defined by the steps of the HTA process (When?), the objective of involvement (Why?), the type of people involved (Who?) and the mechanisms of user involvement (How?). The different steps of the HTA process have been grouped together in three main stages (When?). The first stage is the selection of evaluation topics and comprises the suggestion and prioritization of topics. The second stage, evaluation, includes several steps: protocol development, review of evidence, contextualization or collection of primary data (if relevant), analysis and synthesis of results, final report, and recommendations. The communication and dissemination of results represents the third and last stage of the HTA process. Regarding the people involved (Who?), a distinction was made between “specialists” and “generalists” similar to that made by other authors between patient and public (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis7). Specialists refer to health-service users (or patients), namely people who are, or have previously been, users of health services directly affected by the technology that is evaluated. User family members and user representatives, such as community groups that represent user interests, are also included in this category. Generalists include people who represent all potential or current health-service users, but who are not related to the specific health condition targeted by the assessment. Mechanisms of involvement (How?) included the levels of user involvement and activities for involving health-service users. The three levels of user involvement presented in the framework correspond to those defined by Rowe and Frewer (Reference Rowe and Frewer9): (i) Information, related to the communication of HTA results to health-service users; (ii) Consultation, which includes different ways of asking users about their values, perspectives, needs, and/or preferences to feed the different phases of the HTA process; and (iii) Participation, which includes different mechanisms that could be put in place to actively involve health-service users in the HTA process.
This framework was used to involve service users (or their families) in a specific HTA of alternative measures to restraint and seclusion among adults in short-term hospital wards (mainly in psychiatry) and long-term-care facilities (for the elderly) (Reference Gagnon, Gagnon and St-Pierre11). In this HTA, restraint includes several measures to restrict a person's freedom of movement using physical force or mechanical devices. Seclusion, a type of restraint, involves confining a person in a room from which the person cannot exit freely.
This topic was prioritized by the HTA Roundtable of Université Laval's Integrated University Health Network (Quebec, Canada) that mandated the hospital-based HTA unit (HTAU) of the CHU de Québec to carry out the assessment with its partners. This study focuses on one of the objectives of a larger study (Reference Gagnon, Gagnon and St-Pierre11): to explore the applicability and relevance of our framework for user involvement in this specific HTA according to stakeholders concerned by the topic.
METHODS
Study Design and Participants
We conducted a qualitative study to explore the points of views of people working in the sectors concerned by the assessment topic (i.e., mental health and geriatrics or long-term-care facilities) regarding the applicability and relevance of our framework for user involvement in this specific HTA. We used a purposive sample to recruit study participants in the catchment area of the HTA Roundtable. We asked the chief executive officers (CEOs) of the seven regional health agencies included in this catchment area to refer us to key individuals working in these sectors. We contacted the key individuals to ask them to participate in an interview. Key user representatives in the mental-health field were also recruited through community groups that collaborated in other components of our project. Because it was difficult to find users’ associations in the sector related to long-term-care facilities, this group was not be included in the study. The project received ethical approval from the CHU de Québec's ethical review committee, and all interviewees signed a consent form before their interview.
Data Collection
We conducted individual semi-structured interviews with participants to explore their point of view concerning the applicability and relevance of the framework for user involvement in HTA (Table 1) to the specific context of the assessment of alternative measures to restraint and seclusion.
The framework and interview guide were sent to study participants a few days before their interview. After a short presentation of the framework (Table 1) by the interviewer, the interview included questions about the applicability and relevance of the different models of user involvement presented in the framework in the specific context of the assessment of alternative measures to restraint and seclusion. Specifically, we asked questions about the perceptions of different stakeholders concerning different levels of user (or patient) involvement (information, consultation, participation) at different stages of the HTA process applied in their field (mental health and/or geriatrics), as well as the types of participants and the activities they see as possible. We also asked study participants to share their general appreciation of the framework.
Data Analysis
The interviews were audio taped, transcribed verbatim, and analyzed using N*Vivo 8 software (12). We performed a thematic analysis using the method described by Huberman and Miles (Reference Huberman and Miles13), which includes three stages in the qualitative analysis of data: data reduction, data display, and drawing conclusions. We coded data according to predefined dimensions from the conceptual framework for user involvement (Table 1), that is, according the different steps of the process (When?), the pertinence of involving users and the purposes of the involvement (Why?), the people involved (Who?), and the level of involvement and the activities proposed (How?). Two members of the research team proceeded independently to codify the first four interviews and sought consensus for the final codification to ensure the internal validity of the analysis process.
RESULTS
Participants
In total, twenty people participated in an interview from September 2011 to late January 2012. Study participants were predominantly involved in the mental-health sector (n = 10), including five managers or coordinators, and five representatives of user or community groups. Five interviewees were involved in geriatric and long-term-care facilities as managers, coordinators or clinicians. The last five were high-level managers not associated with a particular sector. The interviewees represented five of the seven health regions of the University Integrated Health Network and included the most populated areas. The interviews were conducted face-to-face, with the exception of three conducted by telephone due to geographical distance. Interviews lasted on average 72 minutes (35 to 90 min).
Overall Appreciation of the Framework
Generally, interviewees agreed that a greater emphasis should be given to user perspectives in their respective areas, for various reasons: to have the point of view of those who receive care and benefit from “service users’ expertise”, to better meet the needs of users, to favor their empowerment, to consider all perspectives to foster informed decision making, and ultimately, to improve quality and safety of care.
Most interviewees found the framework useful as a means to inform and raise awareness among decision makers and practitioners concerning different options for user involvement in HTA. It was also seen as likely to help in structuring various activities of user involvement at the local level.
For their part, user representatives liked the fact that the framework shows potential for user involvement in all stages of the HTA process. A quarter of the respondents found that the framework required explanations to be understood and some noticed that it is a theoretical rather than a practical tool.
Results reported below and summarized in Appendix 1 present the main perceptions of interviewees concerning the relevance and applicability of different models of user involvement proposed in the framework with respect to the assessment of alternative measures to restraint and seclusion in psychiatric and long-term care. In the context of long-term-care facilities, involving the family members of residents was discussed instead of involving service users themselves, given the cognitive problems of people likely to experience restraint or seclusion. The results are presented in relation to the three stages of the HTA process (selection of topics, evaluation, and dissemination).
User Involvement in the Selection of HTA Topics
Steps for involvement (When?)
Interviewees generally agreed that users of mental-health services or relatives of residents in long-term-care facilities should be involved from the beginning of the HTA process, starting with the step of suggesting an evaluation topic.
User involvement in the prioritization of topics was considered less clearly appropriate for some respondents. As observed by an interviewee, it is difficult to ask the service users to be objective in choosing priorities. Furthermore, the process of prioritizing HTA topics respond to well-defined criteria, and left little room for the user perspectives, as noted by other participants:
Yes, the question is always what do you do with the answer! If we had the intention of making such an assessment rather than another, or to initiate one . . . finally we are told that this one [was prioritized by clients] . . . (Manager – Not Associated with a particular sector (M-NA)).
Relevance and Purposes (Why?)
A majority of interviewees saw a place for users of mental-health services or relatives of residents in long-term-care facilities in the choice of evaluation topics to reflect user needs, concerns, and values.
Often we will decide things in relation to our values, our experience. They experience completely different things. In the same way, on issues of assessment . . . there are subjects that are almost taboo. The example of medication . . . the impact it will have on sexuality, for instance. Is this a topic that interests them more maybe than we are comfortable talking about (Manager –Mental Health (M-MH)).
Type of Participants (Who?)
The “specialist” service users, who are people directly affected by the technology evaluated (including their representatives or their families), were perceived as the most suitable participants to be involved in the selection of HTA topics in a specific field. The possibility of involving “generalists,” such as members of hospital user committees or service users with chronic diseases to prioritize topics in different areas was also mentioned by a few respondents.
Levels of Involvement and Activities (How?)
Consulting community groups about potential topics for assessment was the main strategy proposed in the mental-health field. As pointed out by a manager, it may, however, be difficult for users to propose topics without preparation. As one study participant observed, the development of relationships between the research community and these community groups could be an effective strategy to collect evaluation topics among healthcare service users:
But I do think that it could be important [to involve users at this stage]. This presupposes the development of a certain complicity between the research community and people in the field [. . .] A gradual and effective involvement of service users will emerge at some point; it goes without saying that they will suggest topics (M-MH).
Organizing public meetings with service users and providing a list of potential topics to structure the approach were also suggested by other participants.
User Involvement in the Evaluation Stage
Two mechanisms of user involvement were discussed during the interviews concerning the phase of the assessment itself: consultation of users to inform the evaluation and their active participation in the evaluation process.
Steps for Involvement (When?)
Many respondents believe that user representatives should participate as much as possible in all steps of an assessment. They should therefore consider themselves full participants and familiarize themselves with the HTA process, as well as “with the reason why they are consulted” (M-MH). But this perception was not shared by all interviewees; some of them saw user participation as relevant in the discussions involving patient issues, but less so in other more technical issues.
Involving user representatives at the beginning of the process in the development of the evaluation plan (identification of the dimensions to be evaluated, the criteria to be considered for assessing the validity of an intervention, etc.) is particularly beneficial, according to many interviewees.
User representatives also stressed the importance of involving them in the preparation of a consultation of service users, more specifically in recruitment and in the development of tools for collecting data from users:
When you do research, it's all in the question that you ask. For example, “Madam, is the food good?” People will say yes. But if I say, “Is that the food that you like?” or “Is that the food that you want? “. . . we won't have the same answers. For me, it's important to include users from the outset, even if they don't have scientific knowledge [. . .] There is a tendency now to include them once the work is done by saying “Does this work for you?”. (Service users-Mental health (SU-MH)).
Consulting service users seems particularly relevant in the mental-health and geriatric fields, according to interviewees. In these areas, service users often have to live with technologies for a long time or on a frequent basis. In addition to alternative measures to restraint and seclusion, some examples of relevant topics were given during the interviews (e.g., psychiatric drugs, meals, and daily care in long-term-care facilities).
Participation of Users in the HTA Process (How?)
It is generally the practice in HTA in Quebec at the local level to form a multidisciplinary working group to conduct the evaluation. In the specific assessment of alternative measures to restraint and seclusion, a supraregional working group was set up to fulfil the mandate, bringing together a variety of stakeholders concerned by the topic who met at key times during the evaluation process. According to interviewees, one of the primary ways in which service users could be involved in HTA is through their direct participation in these working groups.
Consultation Methods and Activities (How?)
Qualitative methods, and more particularly focus groups, were favored as consultation strategies because of the wealth of information and nuances that could result from interactions among participants. Interviewees, mainly user representatives, thought that focus groups facilitated or co-facilitated by a person who has experience related to the topic (such as a representative of a community group or a peer worker) would be a key element of success for consultation in the field of mental health. All interviewed user representatives also stressed the importance of monitoring and providing feedback for people involved in the HTA process.
A representative of the community setting also suggested the organization of public meetings to consult service users. According to a clinical manager interviewed, it is also necessary to develop approaches to reach vulnerable and illiterate people to ask them about their care experiences:
In an era when much attention is given to vulnerable populations, to those patients who are deprived of almost everything in life, I think we should try to reach those people. I think that we should get closer. . . probably, a kind of street worker or the equivalent of a social worker who's going to talk with people and discuss their care experiences [. . .] (M-NA).
Relevance and Purposes (Why?)
The direct participation of user representatives in the HTA process, through their involvement in working groups for example, is seen by many managers and coordinators interviewed as a direct way to get their perspectives during the discussions and their feedback on HTA results. User representatives found their participation in working groups or committees very relevant because service users and their representatives bring experiential knowledge that is different from that of clinicians or managers:
It's different to talk about restraint when you’ve never experienced it. Just go to a psychiatric ward, with the doors locked. . . you don't have your freedom. It is a terrible shock. [. . .] The anguish that we experience is indescribable. We must constantly remind people (SU-MH).
According to user representatives, in addition to the direct contributions users can make in HTA, their participation could bring more credibility to an evaluation for other users and could help them use the results with a greater degree of confidence:
This is richness, this is a plus, and for those who’ll receive care with the technology that has been evaluated, the fact that there were several peers who participated in the evaluation can reassure them (SU-MH).
In turn, user participation in HTA could enhance the users’ self-esteem. It also allows them to better understand healthcare institutions, including their obligations and underlying constraints, as observed by some managers.
Types of Participants (Who?)
According to most interviewees, the basic criteria for selecting user representatives to participate in a working group are being involved in the community related to the topic (e.g., a community group) and having experience as a service user, while having enough distance from the experience related to the technology evaluated. The person should also exhibit good teamwork, openness, and communication skills.
Most interviewees mentioned community group representatives as relevant participants in an HTA working group. They are seen as having a good knowledge of the issues and experiences of the members of their organization and as able to easily get information from them. Also, “they are highly trusted by the people they represent” (M-NA). According to a user representative, community-group representatives have legitimacy because they have been elected by group members. They have the duty to represent the members of the organization rather than just express a personal point of view.
However, some interviewees noted a possible drawback regarding the participation of representatives of activist organizations: they can use these working groups as a lobbying platform for their organizations. This attitude could affect the smooth functioning of HTA working groups:
If they come here to defend their point of view, it's like a lobby. . . This gets in the way of developing ideas. [. . .]Their minds are often made up. If most discussions, rather being exchanges of information, involve defending a point of view and presenting arguments for and against, then I’m not sure that the process is very productive (M-NA).
Other user representatives have been put forward as possible participants in a working group in a mental-health setting, mainly peer-support workers who have the distance and expertise needed to fulfil this role. A peer-support worker is a person with lived experience of a mental illness who has gone through a process of recovery and could help and support other people with mental-health issues in their own recovery process.
When we explored the possibility of active participation for the families of residents in long-term-care facilities, interviewees were rather open to the idea. However, this possibility remained more theoretical than for mental healthcare users who are more organized and mobilized within associations and already sit on committees that are working to improve the organization of health services.
People directly affected by the technology (“specialists”), or their family, when users have suffered a loss of autonomy, are those who should be consulted in the data collection process. Witnesses or other patients in the environment of the person directly affected by the technology, for instance patients in a psychiatric ward, could also be consulted, even if they have not experienced restraint or seclusion themselves. Some participants mentioned other criteria to take into account in the recruitment of participants, including having the cognitive ability to participate in these consultations and having taken enough steps backward from their experience as related to the evaluation topic.
Dissemination of HTA Results to Users
Relevance of Informing Service Users about HTA (Why?)
All interviewees agreed on the importance of informing mental-health-service users and families of residents in long-term care-facilities about the results of an HTA. In the case of this specific HTA, service users or their families should be informed about the effectiveness of alternative methods to restraint and seclusion to give their informed consent regarding their usage and to promote the implementation of recommendations.
Often, and increasingly, families of residents in long-term-care facilities are involved in decision making. So, if the alternative measure is 75 percent effective, they will perhaps accept it more than if it were 15 percent effective. [. . .] In the process of implementation of alternative measures, these people must be informed because they have to give their consent (Clinician-Geriatrics (C-G)).
In addition to supporting informed decision making, information for users will enable them to better understand interventions and services they receive and to judge them more properly:
The results absolutely have to be presented so that people are informed. They need information to make free and informed decisions and appropriately assess the services they received. [. . .] For example, someone who doesn't know how it should happen could be abused, but they won't know it, because they don't know how this should happen. Or, conversely, they can whine and criticize, but in evidence-based practice, that's what they say to do [U-MH].
Activities of Communication (How?)
Different means of communicating HTA results were mentioned during the interviews. A brochure or leaflet summarizing the key points of information was mentioned by almost half of the respondents. User representatives favored in-person meetings (e.g., meetings combining information and consultation). The use of information technologies (Web site or DVD) was also suggested as another avenue to explore that may be especially effective, according to some participants.
Involving Users in the Communication and Dissemination of Results
Purpose (Why?), Participants (Who?), and Mechanisms (How?)
A more proactive strategy to favor the dissemination of HTA results to users is to consult or to involve them actively in the development of information materials and dissemination strategies. Collaboration with community groups that play the role of “transmission belts” was mentioned as the main knowledge-transfer strategy to reach users. An interviewee noted, however, that other means were needed to reach service users who are not in these groups, for example, through community workers.
The active participation of user representatives in HTA could also enable appropriation and dissemination of results during the action. Besides, user participation in the dissemination of HTA results could help inform researchers about the needs and priority research topics for users, thus “closing the loop” from research to practice (M-MH).
Barriers and Facilitators Related to Service-User Involvement
Barriers and facilitators related to service-user involvement according to interviewees are presented in Tables 2 and 3. We highlight the main themes here.
Barriers to User Involvement in HTA in Psychiatric or Long-Term Care Facility Fields According to Level of Involvement
Facilitators to User Involvement in HTA in Psychiatric or Long-Term Care Facility Fields According to Level of Involvement
Barriers to Service-User Involvement
One of the main barriers to user involvement noted by interviewees is the top-down health system, which takes little account of the user's perspective.
Other barriers are related to the recruitment of service users for a consultation as well as for direct participation in the HTA process. Participants could be difficult to recruit because of the sensitivity of the topic or the fear of stigma attached to a field such as psychiatry. The lack of awareness of certain HTA topics by both users and their relatives, because these topics are “concepts of clinicians or managers” was also pointed out pointed by a manager.
[. . .]alternatives measures to restraint and seclusion is a concept of clinicians and managers, not a concept of service users. Despite being very aggressive, a user will spend 15 days in a hospital, never in seclusion and without undergoing restraint, and never know why. [. . .] It's only because we’ll have properly applied the alternative measures (M-MH).
Concerning family-member involvement, a lack of knowledge of the evaluation topic and of the reality of the service user, a lack of availability to become involved, and an inability to step back regarding the situation of their relatives were mentioned. A lack of motivation to get involved in a public function about what is often perceived as a private or family situation was also mentioned as a barrier to recruitment.
The active participation of service users in the HTA process, for instance in a working group, poses important challenges. Some managers noted that “you can't tell service users everything” (M-MH) or that “they lack knowledge and perspective to be able to take part in some decisions”. Motivation and productivity may also be different for user representatives compared with other members of the working group. Regarding the participation of representatives of activist organizations, some interviewees pointed out that they can use the working groups as a lobbying platform for their organization and this can affect the smooth functioning of meetings.
Although the participation of the same user representatives throughout the process seems advantageous because it allows them to become familiar with the HTA process, it can also be very demanding for participants. Some factors could also affect the success of user participation: their lack of preparation, a lack of clarity regarding their roles, a lack of tools, and/or the use of language that is too technical, scientific or hermetic.
Facilitators to Service-User Involvement
Many interviewees mentioned that community groups or associations were the best intermediaries between researchers and service users for the organization of a consultation and the recruitment of user representatives. These organizations are highly trusted by their members, and their cooperation for the recruitment of participants can be an effective strategy.
For user consultation, it may also be a good idea to vary sources of recruitment (e.g., by means of user committees of the healthcare facility, health-services delivery points, or through managers or healthcare providers). User representatives considered that facilitation of the focus group by a representative of a community group or a peer-support worker was a key element of success.
With respect to user participation in a working group, it is important to recruit a sufficient number of user representatives to ensure that their expertise is represented in a similar proportion to that of other experts. Respondents also favored smaller working groups, thus enabling the participation of users to have an impact. User representatives delegated to committees should be provided with initial training, including basic information on HTA, on the specific topic that will be evaluated, and on the basic skills required to participate in a working group.
DISCUSSION
Through interviews with managers, clinicians and service-user representatives, we explored the relevance and applicability of a framework for user involvement in HTA at the local level in a specific assessment related to psychiatric services and long-term-care facilities for the elderly, and also more broadly, in the fields of mental health and geriatrics. However, its applicability to other sectors would need to be assessed. Our results lead us to make certain observations.
First, most of the interviewees considered it essential to integrate users’ experiences and values in HTA. According to the phase of the HTA process, different objectives of user involvement could be targeted, and many interviewees found this involvement relevant to all phases of the HTA process.
Second, the different strategies or mechanisms to integrate users’ values and perspectives presented in the proposed framework were generally seen as relevant and applicable, particularly in the mental-health setting. Most interviewees believed that consulting service users to inform the evaluation was essential. However, some interviewees emphasized that user consultation should be integrated into a more holistic and participatory perspective. They noticed that service users are often consulted for very specific research purposes rather than to participate as real stakeholders. They thought that people directly affected by decisions should be involved alongside other stakeholders (policy makers, healthcare managers, clinicians, researchers, etc.) in shaping public policies. Direct participation allows users to be part of the decision-making process and not only recipients of information. However, if a clear distinction has to be made between the use of information on users’ values and preferences and their direct involvement in the decision-making process (Reference Kelson, Akl and Bastian14), both have their place in HTA at the local level. Restricting involvement to direct participation raises further concerns about the identification of participants who can represent the range of user experience and may eliminate input from marginalized and vulnerable populations (Reference Kelson, Akl and Bastian14). As pointed out in our study, the consultative approach makes it possible to consider the views of a variety of participants who do not necessarily meet the requirements of participation in HTA committees. It is important to find ways to reach users in their diversity, including vulnerable people and those presenting specific challenges, such as cognitive problems to better answer the needs of the population and improve health care.
Third, different types of participants who could be involved in HTA were mentioned during the interviews. Service users, who are people directly affected by the technology evaluated, were perceived as the most suitable participants to be involved in consultation. In care institutions for the elderly, it was suggested that family members represent the residents. For direct participation, representatives of community groups and peer-support workers (in mental health setting) were most frequently mentioned. These different categories of participants can be associated with various facilitators or barriers. For instance, care users may be difficult to reach and mobilize without the support of community or activist groups who are often important intermediaries between users and researchers or HTA producers. However, the participation of representatives of activist groups can be seen as likely to pose significant challenges. Specific barriers to involving families of users have been reported in this study. The family does not necessarily have the same motivation to participate in HTA as the users themselves. Furthermore, other studies have shown that family members could be in conflict with patients’ interest (Reference Entwistle, Renfrew, Yearley, Forrester and Lamont15) or have a high degree of inaccuracy in determining or predicting the patients’ wishes or expectations (Reference Shalowitz, Garrett-Mayer and Wendler16). Our results support the conclusion that involving users (or their representatives) and involving their relatives should be considered as two quite distinct experiences.
Fourth, in addition to the barriers related to the recruitment or the participation of different types of participants, one of the main barriers noted by interviewees is the top-down health system, which takes little account of user perspectives. Many authors stressed that a better evidence base is needed to convince providers and managers to place greater emphasis on users’ views when making decisions about services. According to a Cochrane systematic review (Reference Nilsen, Myrhaug, Johansen, Oliver and Oxman17), the only evidence (although of moderate quality) to support the effectiveness of involving users concerns patient-information material, which is more relevant, readable, and understandable to patients when they are involved in its development.
As evaluating user-involvement experiences in HTA remains a priority to provide a better evidence base, we have to consider alternative approaches to experimental research and to synthesize diverse sources of evidence (Reference Craig18). For instance, the analysis of factors that contribute to the success of different experiments with user involvement could enhance our understanding of the best methods to involve users (Reference Craig18).
Furthermore, these evaluations face important conceptual and methodological challenges, from the definition of user involvement to the measurement of its effects. Healthcare professionals and service users understand and practice user involvement in different ways based on individual ideologies, circumstances, and needs (Reference Fudge, Wolfe and McKevitt19). The simple question of the choice of outcomes as indicators of the success of user involvement depends on the different perspectives of the key stakeholders (Reference Fudge, Wolfe and McKevitt19).
LIMITATIONS
A limitation to this study is the influence that could result in the reference framework being presented to the participants beforehand. A social desirability bias may have been present and accentuated a participant's favorable perception of user involvement in HTA.
Another limitation is the smaller representation of the long-term care setting: only five managers were specifically associated with this area, and the community sector was not represented among study participants. Despite our efforts, we were not able to recruit community organization representatives in this sector. This contrasts with mental-health-service users who are organized in many community groups. In their study of user involvement in health-service development, Fudge et al. (Reference Fudge, Wolfe and McKevitt19) experienced the same difficulty: stroke-service users were not organized into activist movements such as those for HIV/AIDS, maternity services or mental-health users. Our results also support the idea that the existence and characteristics of patient groups make user involvement relatively difficult to achieve in some instances.
CONCLUSIONS
This study highlighted the relevance and applicability of a framework for user involvement in HTA at the local level in the specific assessment of alternative measures to restraint and seclusion in short-term psychiatric care and long-term-care facilities. This framework could help advance knowledge on user involvement in the different steps of HTA at the local level by making practitioners and health managers aware of the possibilities of different approaches and providing a structured way to classify and describe strategies. However, more structured tools are necessary to help managers concretize these possibilities, and a better evidence base is needed to persuade providers and managers to place greater emphasis on users’ views when they make decisions about health technologies and services. To develop this base, alternative approaches to experimental research have to be developed to evaluate complex social interventions and to synthesize diverse sources of evidence.
CONFLICTS OF INTEREST
The authors declare that they have no competing interests.
