Recruitment and Focus Groups

We recruited participants for focus groups to explore patients’ experiences taking dabigatran (Pradaxa) for the prevention of clots due to atrial fibrillation or venous thromboembolism and their perceptions of information summaries intended to aid with shared decision-making about anticoagulants. The study was advertised on the Facebook page of a US national patient organization for atrial fibrillation. Interested individuals completed a RedCap screening form that minimized required identifiers (name, email, age range, medication, and diagnosis).Reference Harris, Taylor and Minor ⁹ Individuals were eligible for focus groups if they were 65 years or older and currently taking dabigatran, although the form also asked about other direct oral anticoagulants (DOACs) and warfarin (Coumadin) use. We received over 300 expressions of interest in one week, and, pleased by the broad and passionate interest in our research topic, decided to randomize invitations to the focus groups; eligible respondents available at pre-specified times were randomized, and the first twelve were invited to participate (Figure 1). At the start of each focus group, participants were informed the session would be recorded and transcribed, that they were free to leave, and remaining signified consent; no participants left. The study was approved by the Mass General Brigham institutional review board.

Figure 1.

Recruitment of participants to focus groups.

VTE = venous thromboembolism; DOAC = direct oral anticoagulant (including apixaban, rivoraxaban, edoxaban, dabigatran).

*An additional participant joined the focus group saying that a friend had shared the invitation with him and asking repeatedly to stay (later review found that this participant also completed the screening form).

As the focus groups proceeded, we began to suspect that the participants were impostors who did not meet eligibility criteria despite claiming otherwise, which we explain below. We did not confront participants and compensated all for their participation. We then revised our recruitment strategy. First, given low dabigatran use among US Medicare patients, we expanded eligibility criteria to include all DOACs — apixaban, dabigatran, rivaroxaban, and edoxaban — to increase the pool of eligible potential participants. Next, we used two research recruitment sites rather than social media: ResearchMatch, a national volunteer registry, and Rally, a local portal hosted by Mass General Brigham. ¹⁰ Third, we revised the screening form to include more personal details, like birth year, state of residence, and profession to cross-check with public records. Despite the broadened criteria, only 27 individuals responded, and all eligible respondents were invited to participate (Figure 1). Other procedures remained unchanged.

The focus groups followed a semi-structured guide and began with questions about participants’ decision to initiate a DOAC and experiences taking it. Next, participants were shown four successive novel displays of evidence comparing dabigatran to warfarin, and after each presentation, they were asked how the information would influence their medication choice. Focus groups were recorded, transcribed, and coded using thematic analysis to construct themes.Reference Braun and Clarke ¹¹ Full results of the study with the better-screened patient participants are reported elsewhere.Reference Rand, McGraw and Wang ¹²

Thirty-eight individuals participated: 23 in the first two focus groups in November 2022, each lasting 96–98 minutes, and 15 in four focus groups after our recruitment adjustments between March and April 2023 on Zoom, each lasting between 61–76 minutes.

Major Differences in Focus Group Experiences

A number of characteristics distinguished participants in the first two focus groups vs. patients in the final four groups (Table 1). These characteristics have been previously reported, and so our study adds to a growing literature addressing these signals that a participant is an impostor. With respect to recruitment and administrative communications, participants in the first two focus groups responded quickly to the original advertisement and continued to respond quickly to study-related preparatory emails. These emails all came from the same free service (all had Gmail accounts) and used the same or similar 1–2 sentence responses with similar formatting and no salutation or signature/name appended. ¹³ Participants in the first two groups were also excessively attentive to payment after the focus group, some sending an email inquiry about the payment status each day. ¹⁴ We also found that participants in the first two focus groups continued to email the researchers after the focus group asking about future research opportunities or in one case to request to participate again in the focus group in which they had already participated. These behaviors differed substantially from the patients in the final four focus groups, who signed up much more slowly to the focus groups, responded from a variety of email services, varied in the format and tone of their emails, were much less likely to proactively contact the research team, and were far less focused on honorarium payment.

Table 1.

Impostor participant characteristics. Indicators identified in the first two focus groups compared to later focus groups

The conduct of the first two focus groups also proceeded far differently from the final four groups and raised suspicions about study eligibility. Nearly all participants in the first two focus groups (18/23, 78%) did not turn on their cameras, as has also been noted by other researchers. ¹⁵ Investigators conducting the focus group (LZR, SM) independently concluded that those who did turn on cameras did not look sixty-five or older, including one shirtless participant — a subjective alerting judgment similar to those reported in other studies of impostors in research. ¹⁶ Consistent with the investigators’ suspicions, one participant stated he was twenty-six years old, and on being informed he was therefore ineligible, he then claimed he was actually sixty-six. Eight of the twelve explanations offered during the first two focus groups for why participants take dabigatran contradicted screening form responses. For example, a participant who claimed a diagnosis of venous thromboembolism in the screening form then explained during the focus group that he took dabigatran for atrial fibrillation, while three participants selected atrial fibrillation in the screening form and described taking dabigatran to prevent strokes following an accident.

We also observed signs that participants were in the same room or acting as multiple participants simultaneously. Two participants with video shared the same distinctive wall decorations. For a third participant, the focus group could be heard from another, off-screen device when unmuted. A participant in Focus Group 2 initially joined with the same name as a participant from Focus Group 1 before changing Zoom names. Impostor participants were slow to respond to their names when called upon. In one example of confused identity, Participant #8 introduced himself as a different participant (“hello, this is Participant #15”) contradicting his self-assigned Zoom name. Both Participants #8 and #15 responded to the other person’s name at least once during the session. Some impostor participants seldom participated orally (two spoke briefly just once during a 90-minute session; another spoke only twice saying “hello” and then claiming when called on that he had already shared a response to a question); one participant never spoke, using only the chat, so we could not rule out one person using multiple devices to join the focus group.

Thin engagement also characterized the first two focus groups. The facilitator had to repeatedly ask participants to share their views and experience, which were stated straightforwardly and rarely in response to other participants. Participants provided little personal background, and they did not relate new information shared with them back to their personal situation or concerns (Table 2). Instead, they restated evidence or used it to draw a conclusion, as in “It is clearly stated that Coumadin has lower risks to Pradaxa that is in terms of clotting, then if you look at the bleeding effect you will see that here you go for Pradaxa instead of Coumadin. So in summary, I believe Coumadin has a lower risk when it comes to a lung clot or deep vein clot, and when it comes to major bleeding, I believe everyone should go for Pradaxa … I’ll go for Pradaxa.”

Table 2.

Impostors in the first two focus groups vs. patients in the later focus groups. Illustrative quotes from each group of experiences with anticoagulants. Participant number is in parentheses

By contrast, in the final four focus groups, all fifteen participants had their cameras on and asked questions of each other and the facilitator. Conversation flowed freely. In terms of the content of the conversation, participants discussed the evidence in relation to their own experiences and asked what it meant for them. For example, multiple participants raised questions about who had been in the study populations and whether the study population was generalizable to them, as would be expected of patients who were actually taking a medication for an actual medical issue affecting them.

We drew a few conclusions from these focus group experiences. First, most of the characteristics, particularly those related to screening and communications, are not individually determinative of an impostor. When multiple characteristics are present, either in one individual or across the group, researchers should be on guard for potential impostors. Second, our experience suggests that impostors may have circulated the signup link and colluded together as multiple participants. Advertising research on social media can promote interest among eligible participants, but makes it even easier for impostors to find and share the study sign-up, so investigators must exercise even greater scrutiny when screening. While we cannot be absolutely certain in our identification of impostors and actual patient participants, the presence of multiple indicators increase our confidence in identifying impostors. It may be that the patients we considered to be eligible were actually much more skilled impostors.

Effect of Different Focus Group Experiences on Findings

Impostor participants can impact findings in important ways. Discussion in our focus groups addressed clinical outcomes and knowledge about DOACs. The most common adverse event mentioned in the first two focus groups was gastrointestinal symptoms; only two participants mentioned minor bleeding. Nearly half (11/23, 48%) of impostors reported gastrointestinal side effects, like severe stomach aches, and 4 (17%) reported serious adverse events (hematemesis or bloody stools). Only 2 (9%) impostors commented on non-gastrointestinal bleeding while on dabigatran and none raised concerns about bleeding from injuries, like small cuts or blood draws, head injuries, or reported that their doctors had warned them of this risk. Only 2 (9%) impostors expressed awareness of warfarin as an alternative. Impostors reported gastrointestinal side effects at a greater rate than reported in clinical trials of dabigatran.Reference Bytzer ¹⁷ We therefore conclude that it is not good for a person’s health to pretend to take dabigatran.

By contrast, a majority of patients taking DOACs (9/15, 60%) described experiences with minor bleeding, like cuts that take a long time to heal, and bruises. All patients taking DOACs worried about major bleeds and head injuries, leading several to stop physical activities, like bicycling and pickleball. Some patients taking DOACs who had experienced minor bleeding or bruising claimed to experience no side effects, suggesting that they viewed these experiences as normal parts of anticoagulation therapy. All but one patient taking DOACs (14/15, 93%) had taken warfarin or had a family member/friend who did, and perceived warfarin to cause more bleeding and bruising, in addition to imposing other restrictions on their lives.

When shown a direct-to-consumer television advertisement for dabigatran as part of a discussion on evidence for informed decision-making, patients taking DOACs universally questioned the value of the TV ad for dabigatran. A few noted it might encourage them to ask a doctor about alternatives to warfarin, if they were on it. Patients taking DOACs noted that the ad listed side effects too quickly, with one observing, “well, if you take this medicine, you’re going to have diarrhea and possible death and rashes, and all this stuff… they’ve kind of become a joke.” (Patient #30) These participants also worried about the high out-of-pocket costs of their medications.

By contrast, impostors viewed the ad positively; adjectives used included “informative”, “educational”, “precise”, “meaningful”, “enlightening”, and “comprehensive.” Only one impostor expressed a reservation that the ad made “warfarin look bad.” (Impostor #12)

One area of concordance across all groups was trust in their doctors’ recommendations, although patients taking a DOAC emphasized the need to also do their own research (Table 2).

The differences between the results of the two sets of focus groups shows the importance of recruiting valid patients. Any one of these findings could impact recommendations for doctor-patient communication and the usefulness of patient information materials that might emerge from the focus groups.

Recommendations

Our experience shows the risks of limited screening of participants during study activities and can support the identification of additional red flags qualitative researchers should consider among study participants (Table 1). Qualitative research reports should clearly report their screening strategies to enhance the reliability of their study findings.

During screening, researchers should collect more detailed information to verify the identities and existence of interested potential participants, such as birthdate and address to check against public records. Scheduling could also be done via phone, which adds one barrier to repeat participation by impostors. ¹⁸ Another approach is asking for a photograph of a labelled pill bottle with the participant’s name on it. Recruiting from sites like ResearchMatch rather than social media result in vetted participants but limit the diversity of the potential sample; if recruitment must be done through social media, investigators should be extra attentive and scrutinize enrollment questionnaires for inconsistencies. Less onerous and more inclusive processes encourage more potential participants but also lower barriers to impostors — a tension between the values of open, diverse and valid research participation.

Once researchers are concerned that a participant may be an impostor, they are faced with a dilemma. Challenging a participant in a group setting may chill the focus group discussion. In addition, researchers should be open to the possible range of experiences and communication styles of participants; suspicion about characteristics of impostors may be overly biased and reflect researchers’ expectations rather than participants’ reality. ¹⁹ It may be better, therefore, to use more robust screening methods.