Introduction
In the USA, older adults have a higher rate of suicide than any other segment of the population. Between 1999 and 2005, over 5,000 older adults died by suicide, and men accounted for approximately 85% of these suicides (Centers for Disease Control and Prevention, 2008). There is an urgent need to develop and evaluate suicide prevention strategies for older men. However, such research is difficult to implement partly due to challenges associated with recruiting older suicidal men. Little attention has been given to identifying effective recruitment strategies for such research.
Older adults are more difficult to recruit compared to younger cohorts because of purported difficulties with transportation, increased rates of physical co-morbidity, and poor service utilization (Wilson and Webber, Reference Wilson and Webber1976; Arean and Gallagher-Thompson, Reference Arean and Gallagher-Thompson1996; Adams et al., Reference Adams, Silverman, Musa and Peele1997). Men, and in particular older suicidal men, have less positive attitudes towards help seeking compared to their women counterparts, and thus underutilize mental health services (Salib and El-Nimr, Reference Salib and El-Nimr2003). Therefore, they are exposed less frequently to research participant opportunities. Consequently, there is a need to identify recruitment strategies that are effective for this population. This study presents an overview of the issues and outcomes associated with five recruitment strategies employed in an intervention study for suicidal older men.
A number of suggestions have been offered for recruiting participants for intervention research (Ross et al., Reference Ross, Grant, Counsell, Gillespie, Russell and Prescott1999; Cassidy et al., Reference Cassidy, Baird and Sheikh2001; Veitch et al., Reference Veitch, Hollins, Worley and Mitchell2001; Leonard et al., Reference Leonard, Lester, Rotheram-Borus, Mattes, Gwadz and Ferns2003). Patterson et al. (Reference Patterson, Mairs and Borschmann2011) suggest that recruitment involves three phases – set up (identifying and contacting gatekeepers, obtaining their agreement to refer patients), alliance (e.g. engagement with referral sources), and referral mechanisms (e.g. developing referral protocols that fit practice routines) – all of which, the authors argued, are dependent on researchers’ knowledge of the practice culture in which referrals are sought, their interpersonal and engagement skills, and their persistence in approaching potential gatekeepers.
However, very little information is available on whether such approaches are applicable for older adults who are suicidal. Studies on late life suicide prevention or prediction (Morrow-Howell et al., Reference Morrow-Howell, Becker-Kemppainen and Judy1998; Reynolds et al., Reference Reynolds1999; Duberstein et al., Reference Duberstein, Conwell, Seidlitz, Denning, Cox and Caine2000; Unützer et al., Reference Unützer2002; Reference Unützer2006; Bruce et al., Reference Bruce2004; Osvath et al., Reference Osvath, Kovacs, Voros and Fekete2005; Lapierre et al., Reference Lapierre, Dubé, Bouffard and Alain2007; Dombrovski et al., Reference Dombrovski2008; Heisel et al., Reference Heisel, Duberstein, Talbot, King and Tu2009; Wiktorsson et al., Reference Wiktorsson, Marlow, Runeson, Skoog and Waern2011) have tended not to provide information on the challenges, success, or efficiency of such strategies. For example, these studies have not provided an indication of the difficulties or solutions associated with their recruitment protocols. Thus, no guidance is provided for optimizing recruitment practice. Further, with the exception of Unützer et al. (Reference Unützer2002; Reference Unützer2006), studies have not compared the success – that is, the number of enrollments obtained – by different recruitment strategies for the same population. Thus, the method that is likely to result in most enrollments is yet to be identified for this population.
In addition, there is little information available about the efficiency of recruitment strategy across these studies – that is, the yield (i.e. proportion of enrollments to referrals) and costs (i.e. expenses and time) of that strategy. Given that recruitment activities involve an investment of finite resources, it would be important for researchers to have information about the costs and benefits of the deployment of such activities (Ross et al., Reference Ross, Grant, Counsell, Gillespie, Russell and Prescott1999; Patterson et al., Reference Patterson, Mairs and Borschmann2011). Information on the number of people that were approached (or referred) for screening is rarely reported, thus preventing a calculation of yield. Of studies that provided such information, yield rates ranged from 2.8% to 50%. Unützer et al. (Reference Unützer2002) reported that of the 32,908 patients screened at primary care clinics, only 2.8% were enrolled; and of the 2,190 referrals from primary care practitioners or self-referrals, 40.8% were enrolled. Bruce et al. (Reference Bruce2004) reported a yield of 7.4% for primary patients contacted by a letter. Heisel et al. (Reference Heisel, Duberstein, Talbot, King and Tu2009) reported that of the 42 referrals from clinical staff, 50% were enrolled. These studies suggest that clinical referrals result in a greater percentage of enrollments than other methods. However, none of these studies focused exclusively on older suicidal men.
The objective of this study was to evaluate five recruitment strategies for a randomized controlled trial (RCT) investigating the efficacy of a cognitive therapy (CT) for suicidal older men. Participants were randomized to receive approximately 12 sessions of in-person one-to-one CT (the intervention group) or to not receive CT (the control group). The interventions employed for this study have been described elsewhere (Bhar and Brown, Reference Bhar and Brown2012). We report the recruitment methods that were most successful and efficient for the trial, review the difficulties encountered, and outline the lessons learned from using these methods.
Methods
Potential participants
Individuals were eligible for the study if they were male, 60 years or older, and had expressed a desire to kill themselves over the past month. Individuals were excluded from the study if they had medical, cognitive, or psychiatric difficulties that interfered with their ability to participate. A total of 955 participants were considered for the RCT.
Recruitment methods
The 955 potential participants were identified through five recruitment strategies, three of which involved obtaining referrals (from primary care physicians (PCP), psychiatry residents, and the Veterans Administration Medical Center), and two of which involved directly approaching potential participants (sending a letter to primary care patients, and face-to-face meetings with inpatients). Each recruitment strategy, described below, was approved by the university and affiliated health system's Institutional Review Board (IRB).
Referrals from Primary Care Physicians (PCP)
Seven primary care practices affiliated with the University of Pennsylvania were approached for referrals. Physicians were sent an introductory letter and a brief protocol describing the study, and followed up with a phone call. Once the director of the practice agreed to the study, the researchers met with other providers in the practice. Of the seven practices contacted, four were willing to participate as recruitment sources. Physicians were requested to identify patients with suicidal ideation in the past month through questions such as: “In the past month have you had thoughts of taking your own life?” If the patient admitted having such thoughts, the physician was asked to obtain written consent from the patient to allow the researchers to contact the patient with further details about the research. Physicians were provided with a protocol for asking the question about suicide ideation and for obtaining consent. Consent forms were then to be faxed to the researchers by the physician.
Referrals from Psychiatry Residents (Outpatient)
Patients were recruited from a geriatric psychiatric outpatient service of the University of Pennsylvania. Patients scheduled to attend appointments on a given day were identified by the attending psychiatrist. Psychiatric residents approached patients to participate in the study. Research staff met with the patient to then screen the potential participant for eligibility.
Referrals from the Behavioral Health Laboratory of the Philadelphia Veterans Affairs Medical Center (BHL)
The Behavioral Health Laboratory of the Philadelphia Veterans Affairs Medical Center (BHL) is a clinical service of the Veterans Affairs (VA) Medical Center that provides assessments of patients’ mental health symptoms (Oslin et al., Reference Oslin, Ross, Sayers, Murphy, Kane and Katz2006). As part of the BHL assessment, patients were screened for suicide ideation (method described in Paykel et al., Reference Paykel, Myers, Lindenthal and Tanner1974). If the patient reported “thoughts of taking your own life even if you would not really do it” in the past month, he or she was provided a more comprehensive psychiatric evaluation. The psychiatrist informed eligible older male patients about the study. Patients who provided verbal consent to be contacted by researchers were called and screened by the research project director.
Sending Letters to Older Primary Care Patients (Mailing)
After approval was obtained from PCP, researchers obtained a list of all patients who were male and 60 years or above who attended each of four primary care practices in the last year. The list was drawn from a primary care medical record database. Approval for using this database was provided by the University of Pennsylvania IRB. Physicians were then asked to indicate patients from the list whom they did not wish us to contact. Subsequently, the researchers sent the remaining patients a letter, which was pre-approved by the physicians. The letter informed patients of the study and that they would be called by a researcher in two weeks. If the patient did not want to be contacted, they could return a “do not contact” note. Patients who did not return the note were called and screened for interest and eligibility.
Inpatient Psychiatric Unit of the Hospital of the University of Pennsylvania (Inpatient)
After receiving approval from attending physicians and from the university IRB, research staff monitored the electronic tracking system for specific hospital services such as the emergency departments and inpatient psychiatric units. Upon identifying an older adult with suicide ideation, researchers obtained permission from the patient's attending physician to contact the patient (while he was in the inpatient unit) to determine his interest and eligibility for the study.
Screening procedure
Potential participants were informed about the study and screened by phone or face to face for eligibility for the study. Individuals who indicated interest in the study were then screened for whether they met the study inclusion criteria: sex (male), age (at least 60), primary language (English), and suicide desire (scored greater than 0 on the fourth item of the Scale for Suicide Ideation; Beck et al., Reference Beck, Kovacs and Weissman1979). The individuals were also asked questions to determine if they had medical, cognitive, or psychiatric difficulties that would interfere with their ability to participate. For example, they were asked the question: “Do you have any current physical illnesses or disabilities that might interfere with you attending treatment or assessment sessions?” Individuals who were eligible for the study were invited to undergo an informed consent interview and then a baseline assessment.
Results
Of 955 patients approached through these recruitment methods, 281 (29%) were unable to be contacted (e.g. wrong number, individual did not return phone call), 139 (15%) refused contact with researchers, and 302 (32%) refused screening. The remaining 233 patients (24%) were screened. Of these, 185 (79%) did not have suicide ideation (hence, they did not meet study criteria), while 48 patients (21%) were eligible for the study. Of the 48 eligible patients, 15 patients (21%) did not want to participate in the study because they were unable to attend the baseline interview. Consequently, a total of 33 individuals were enrolled in the study (mean age 66.7, SD = 6.5, range 60–87; 64% Caucasian, 36% African American).
As shown in Table 1, the success and yield rate varied across recruitment methods. First, in terms of success (i.e. actual enrollments produced), the BHL produced the most study enrollments (73% of the total study sample), followed by mailings (18%), and outpatient psychiatry (9%). Inpatient psychiatry and PCP strategies did not produce any enrollments. No referrals were received by PCP. Thus, of the methods that resulted in study enrollments, BHL was eight times more successful than inpatient psychiatry, and four times more successful than outpatient psychiatry.
Referrals, screenings, and enrollments by recruitment strategy
PCP = Primary care physicians of the Clinical Care Associate practices of the University of Pennsylvania Health System; Mailing = Direct mailing to older primary care outpatients; Inpatient = Hospital of the University of Pennsylvania Inpatient Unit; Outpatient = The Geriatric Outpatient Psychiatry Center of the University of Pennsylvania; BHL = The Behavioral Health Laboratory of the Philadelphia Veterans Administration; Yield = Percentage of referrals that resulted in enrollments.
Second, the yield (percentage of referrals that resulted in enrollments) was compared across methods. An investigation of yield rates across methods revealed a significant association between the type of recruitment method and whether or not referrals would translate into study enrollments: Fisher's exact test (df = 3) = 117.18, p = 0.000, Cramer's V = .518. Fishers exact tests were used given that expected frequencies were less than 5 in 50% of cells. Degrees of freedom were 3, rather than 4, because the observed frequencies associated with the PCP recruitment method were 0. The BHL and Outpatient methods each produced higher yields than expected by chance (z = 14.8, p < 0.001; z = 3, p < 0.01, respectively), while the Mailing method produced a lower than expected yield, z = –4.4, p < 0.001 (see Table 1).
Third, the time and expenses per enrollment for each recruitment method were calculated (Table 2). Across all recruitment strategies, the recruitment of participants took 273 hours and cost $6,538. Each enrolled patient was estimated to require 8.3 hours of research personnel effort, and to cost $198. The financial expenses associated with each enrollment were based on the amount of time required by the research assistant (salary support) and costs of supplies (e.g. phone calls, printing) for recruitment tasks. These estimates did not include costs associated with activities common across recruitment strategies (e.g. development of recruitment tracking database) or indirect costs such as employee benefits (e.g. health insurance benefits) and costs associated with referral sources (e.g. PCP, BHL research assistant). Using this formula, each enrollment from the BHL strategy was associated with the least amount of time and expense. In contrast, each enrollment obtained through the Mailing strategy was the most time consuming and expensive.
Hours and expenses by recruitment strategy and enrollment
1Abbreviations for strategy are listed below Table 1.
2Salaries ranged from $14 per hour to $32 per hour.
3Supplies include stationary and postage (each letter mailed was estimated to cost $0.68), phone calls (each outgoing phone was estimated at $0.70), and catering (applicable only for the PCP strategy).
4Expenses are the sum of salary and supplies.
5This estimate includes time preparing a submission to the Veterans Affairs (VA) Medical Center Institutional Review Board for permission to recruit patient from the VA.
6NA = Not available because no enrollments were generated.
Discussion
The results demonstrate differences between the recruitment strategies in terms of their success and efficiency. Obtaining referrals from the BHL was the most successful and efficient recruitment strategy. The BHL generated 73% of enrollments for the study, yielded the highest proportion of enrollments, and constituted the least expensive and time-consuming strategy. The PCP and inpatient strategies were the least successful; both generate no enrollments. The mailing strategy was inefficient in terms of the time and expenses associated with each enrollment.
Below, we speculate on the reasons for the superiority of the BHL strategy compared to others employed, and describe the difficulties associated with many of the other recruitment strategies and the solutions implemented. We then offer suggestions for optimizing recruiting strategies for late-life suicide prevention research.
The superiority of the BHL compared to the other recruitment methods may reflect the advantage of the BHL at each stage of the recruitment phase, as framed by Patterson et al. (Reference Patterson, Mairs and Borschmann2011). First, the setup of the BHL as a referral source was formalized through collaboration with the director of the BHL (DO) as a co-investigator of the study. This collaboration may have helped maintain protocols within the BHL for referring patients to the study. Second, the alliance between the university and BHL was maintained, through formal (as described above) and collegial (peer) relationships. Such alliances may have helped maintain the commitment of both teams to resolve barriers to recruitment. Third, the referral mechanism and protocols were framed to fit within the organization structure of the BHL. The routine practice of the BHL was to screen patients for suicide ideation. Therefore, our request for referrals of such patients did not require the BHL personnel to modify their assessment procedures, nor imposed interruptions or changes to the PCP's routine practice. In summary, the success of the BHL at generating enrollments may be due to the effective setup, alliance, and referral mechanisms.
In contrast to the BHL, the method that produced the least number of enrollments were the PCP and inpatient methods. Both methods resulted in no enrollments, and the PCP method produced no referrals. The rationale for recruiting participants from these sources was compelling: High rates of suicide ideation have been identified among older patients in primary care (Lish et al., Reference Lish1996), and other researchers have successfully recruited suicidal older adults from inpatient settings (Pearson et al., Reference Pearson, Conwell and Lyness1997; Duberstein et al., Reference Duberstein, Conwell, Seidlitz, Denning, Cox and Caine2000; Heisel et al., Reference Heisel, Duberstein, Talbot, King and Tu2009). Thus, we anticipated that we would obtain a number of study participants from these settings.
However, we faced three difficulties with these methods. First, although, at face-to-face meetings, PCPs agreed to refer prospective patients to the study, none followed up with this commitment. It is possible that PCPs were not sufficiently engaged to include the referral request into practice. Second, we relied on the physician's assessment of suicidal desire. Such practices were not part of physician's routine practice. At follow-up contact between researchers and PCPs, several PCPs admitted that they forgot to ask patients the screening questions or were reluctant to inform patients about the research for fear of burdening or offending the patient. Several PCPs reported being “too busy.” Some physicians were also concerned about their legal and ethical responsibility should they identify a patient as suicidal. Thus, it appeared that the recruitment effort required from PCPs was perceived by them to be overly discrepant from routine practice, intrusive, and burdensome.
Third, with respect to the inpatient recruitment method, although we did not rely on physicians to identify prospective participants, thus removing the gate-keeping obstacle found with the PCP method, we found that many of the inpatients identified as suicidal and thus potentially suitable for the study were medically unable to participate, or lived out of area and were unable to travel to the research center for follow-up assessments. Of the five older patients identified though our tracking database as having been admitted for suicidal ideation, only two were physically well enough to be screened, and of those, only one met eligibility criteria – but that person lived out of area.
Two solutions were implemented as a result of the poor response rates from the PCP and inpatient methods. First, instead of screening psychiatric inpatients, we began to screen psychiatric outpatients who attended the hospital's outpatient geriatric psychiatry clinic (see Outpatient). The rationale for this strategy was based on several grounds: Older suicidal patients were routinely referred by their PCP to the outpatient psychiatry clinic. Also, our research center was co-located with this clinic thus allowing researchers and psychiatry residents to interact more conveniently through face-to-face meetings. In order to maintain engagement with psychiatry personnel, the research team met with the attending physician and residents every week, and sent weekly email reminders regularly. It is possible that these meetings and communications fostered a relationship between research and clinical personnel, and prompted the residents to remember to refer patients. Further, in contrast to inpatients screened, we anticipated that outpatients would more easily be able to attend research assessment. Eighteen referrals were made to the study, 6% (n = 3) of which resulted in enrollments.
Second, a direct mailing strategy (see mailing) was implemented to overcome the gate-keeping obstacle experienced through the PCP recruitment method. This strategy involved sending a letter to patients in primary care, without requiring the ongoing involvement of primary physicians. As described earlier, we drafted a list of such patients from the health network database, obtained permission from the physician to contact these patients, and subsequently sent a letter to the approved patients. By contacting patients irrespective of their presenting problem, we may have been able to assess a number of patients who may not have been otherwise screened for suicide ideation. Previous research has indicated that suicidal risk in older adults is difficult to detect in primary care because such patients do not readily communicate their suicidal intent to professionals (McIntosh et al., Reference McIntosh, Santos, Hubbard and Overholser1994) and lack a typical profile associated with younger suicidal patients (e.g. history of affective illness or suicidal behavior; Conwell et al., Reference Conwell, Olsen, Caine and Flannery1991).
However, this method raised a number of ethical and logical issues. First, some physicians were concerned about the consequences of a positive screen for suicide ideation. In 97% of cases (59 of 61 PCPs), the physicians were reassured by our risk management protocol. Second, because we obtained contact details of these patients through the hospital database, a large proportion of patients could not be contacted (due to outdated database details) or did not want to be contacted, resulting in a significant loss of potential participants. Only 174 of the 869 patients on our list were screened. Third, because the list contained details of male patients over 60, irrespective of their diagnostic or clinical features, of those 174 individuals screened, only seven were eligible for the study, and six enrolled. Thus, this recruitment method, while successful at producing some enrollment, was expensive and associated with a low yield (0.7%).
The findings from this analysis however are based on several limitations that may be addressed by further research. First, there were differences between recruitment methods that were uncontrolled or measured (e.g. proximity of recruitment site to the research center, size of practice), all of which may have impacted on the success and efficiency between recruitment methods. Future research may explore the extent to which such factors relate to the success and efficiency of a recruitment method. Second, the results of this study are limited to older men with suicidal ideation. Further research is required to explore the extent to which these findings generalize to other older populations. Third, research is required on the efficiency of a broader range of recruitment methods such as media advertisements and public announcements.
Fourth, we did not examine the reasons for why referrals did not amount to enrollments. For example, of the 48 individuals eligible for study, 33 (69%) agreed to participate. The others explained that they had difficulties attending the baseline assessment. It is possible that they had other undisclosed concerns about the research. Reasons for non-participation can be more rigorously examined in future research.
Fifth, the findings from this study may not apply to other countries or systems of healthcare. There are many factors that determine recruitment success, including the enthusiasm of health providers for helping researchers recruit, the amount and type of information provided to recruitment helpers, the relationships they have with researchers, and their availability of time. Such factors are likely to vary across individual recruitment units, and across individuals in these units. Thus, the wider significance of these study findings will need to be explored through local and international studies of recruitment success. For instance, in some countries such as Australia, primary care clinics are not connected to academic networks, thus precluding the use of mailing strategies for recruitment as described above. Conversely, the appointment of joint academic–clinical positions, which currently exist in several healthcare systems, may facilitate research activity, including the recruitment and enrollment of participants within clinical settings.
Two key lessons can be extracted from our experiences, which may help future researchers optimize recruitment strategies for research on suicidal older men. First, the findings of this study have highlighted a role for relationships (e.g. having providers as part of the research team) in facilitating recruitment. The building and maintenance of good working relationships with a referral source appear pivotal for recruitment success. It is unlikely that PCPs or inpatient staff felt engaged with the researchers. Unlike the close relationships built with BHL and outpatient psychiatry registrars, less effort was placed on fostering such collaborative partnerships with PCPs. Thus, future researchers may consider including primary referrers as co-authors, or involving referrers in the design of the study questions and protocols. Such efforts, while time consuming, are purported to translate in more efficient recruitment results.
Second, the study has highlighted a role for processes (fitting in within the usual practice of providers) in facilitating recruitment. We suggest that efficient recruitment strategies for older suicidal adults are dependent on the extent to which referral protocols for gatekeepers are consistent with routine clinical practice. For example, relying on referrals from PCP may be an ineffective recruitment strategy, unless such strategies can be sufficiently engaging for PCPs and are unobtrusive. Researchers may take advantage of collaborative care models for depression that have been introduced in the USA (Katon et al., Reference Katon, Unützer, Wells and Jones2010) and elsewhere (e.g. Wang et al., Reference Wang, Kushner, Frey, Ping Du and Qian2007), where allied health practitioners (e.g. social worker or trained nurse) are co-located in primary care settings to assist patients with depression. Such staff may be more likely and able to engage in collaborative relationships with mental health researchers and improve the screening and referral process. Mental health researchers may choose to seek out a primary care research network that has already integrated these resources.
In summary, recruitment strategies are probably most efficient when both aspects – that is, the integration of recruitment protocols with routine practices, and the presence of relationships with clinical staff – are in place. This study has illustrated the methods that included both aspects were most successful and efficient in recruiting older men with suicidal ideation. There is a paucity of information in the literature on the difficulties associated with recruiting such populations. This study has thus presented an overview of such difficulties and of the solutions implemented to improve recruitment rates. Such information is intended to provide direction to future researchers for optimizing recruitment strategies for studies in suicide prevention with older adults.
Conflict of interest
None.
Description of authors’ roles
Sunil S. Bhar designed the study, supervised data collection, carried out the statistical analysis, and wrote the paper; Shannon Wiltsey-Stirman designed the study and assisted with the writing of the paper; David Zembroski collected the data and assisted with the writing of the paper; Laura McCray assisted with data collection and writing of the paper; David W. Oslin assisted with the collection of data and writing of the paper; Gregory K. Brown supervised data collection and assisted with the statistical design of the study and writing of the paper; and Aaron T. Beck supervised the data collection and assisted with the writing of the paper.
Acknowledgments
This research was supported in part by the National Institute of Mental Health (P30 MH45178 and K99 MH080100) and by The Philadelphia Department of Veterans Affairs (Mental Illness Research, Education and Clinical Center).
