Introduction
Growing attention has been devoted to promoting cognitive health or cognitive vitality among healthy older adults (Martin et al., Reference Martin, Clare, Altgassen, Cameron and Zehnder2011). It is now well-known that healthy behaviors and stimulating physical, cognitive, and social activities can delay the appearance of symptoms related to cognitive disorders (Karp et al., Reference Karp, Paillard-Borg, Wang, Silverstein, Winblad and Fratiglioni2006). Several studies have revealed that group-based interventions were effective in preserving cognitive function of normal aging older adults. Such interventions are usually oriented towards training of specific cognitive functions (e.g. memory, attention, problem-solving; Craik et al., Reference Craik2007), whereas some use strategies to improve self-efficacy towards one's cognitive capacities (Winocur et al., Reference Winocur2007). Other interventions focus on physical activity (Colcombe and Kramer, Reference Colcombe and Kramer2003), stress management (Hayslip et al., Reference Hayslip, Maloy and Kohl1995), or promotion of healthy diet (Small et al., Reference Small2006). Social and intellectual engagement of participants was also included in certain documented interventions (Parisi et al., Reference Parisi, Greene, Morrow and Stine-Morrow2007).
Despite this evidence, few interventions integrated all of these components in a single multifactorial intervention, within a perspective of promoting cognitive vitality among elderly people. Furthermore, most cognitive interventions are typically evaluated in a clinical context (Tardif and Simard, Reference Tardif and Simard2011). Yet, very little is known about the effectiveness of multifactorial programs in improving cognitive abilities among elderly people when implemented in natural settings, such as the community (Rebok et al., Reference Rebok, Carlson and Langbaurn2007).
While conducting research in the community is increasingly needed, it is not without its challenges. Validity, reliability, and objectivity are continually challenged, and the success of a particular intervention in a community-based research may be difficult to prove, given the presence of environmental factors (Israel et al., Reference Israel, Schulz, Parker and Becker1998). Some methodological challenges often raised include: no possible randomization (Bonell et al., Reference Bonell2011), unstandardized implementation of interventions across all sites (Straw and Herrell, Reference Straw and Herrell2002), and high attrition rate (Davis et al., Reference Davis, Broome and Cox2002). While these challenges are inherent to community-based research, many authors believe that evaluative research should be conducted under natural conditions more often for knowledge to be effectively translated into practice (Glasgow et al., Reference Glasgow, Lichtenstein and Marcus2003). Community-based studies provide examples showing that achieving scientific rigor while involving community members’ in the research is challenging but possible (Allison and Rootman, Reference Allison and Rootman1996).
Given these considerations, a team of researchers and practitioners from the Cavendish Health and Social Services Centre (CSSS Cavendish) and Université de Montréal, along with community partners, developed and evaluated the Jog Your Mind program (Musclez vos Méninges in the original French version; Parisien et al., Reference Parisien, Gilbert, Amzallag, Guay, Nour and Laforest2013a; Reference Parisien, Gilbert, Amzallag, Guay, Nour and Laforest2013b). Jog Your Mind is a multifactorial community-based program for promoting cognitive vitality among elderly individuals without known or diagnosed cognitive impairment. Following a literature review and consultations with stakeholders, this cognitive vitality promotion program followed established criteria of quality, namely: (1) specific (addressing frequent complaints of elderly people who have normal cognitive aging); (2) integrative (using a multifactorial approach that combines stimulation of cognitive function, teaching of mnemonic strategies, psychosocial modalities, and strategies to promote lifestyle habits favorable to healthy cognition); (3) educative (based on an andragogy approach in order to pursue objectives related to knowledge, attitudes, and behaviors); (4) community-based (implemented by non-expert group leaders in community organizations); (5) accessible (low cost, adaptable to different settings); and (6) validated (evaluated on its impacts as well as on its implementation process). This program was conceived within a perspective of creating a largely diffusible tool in Quebec (Canada) community settings. It was developed according to the co-construction framework used by the CSSS Cavendish team for the creation of health promotion tools (Nour et al., Reference Nour2010). This framework promotes the engagement of stakeholders, such as volunteers, potential participants, and community partners, from the beginning of the program's development. Steps undertaken to develop the program included: (1) a literature review of existing best practices; (2) building of the program's logic model and creation of the facilitating tools; (3) validation by experts on its scientific content, implementation feasibility, and pragmatic aspects of delivery; and (4) pilot testing with groups of elderly people in order to adjust the program's format and content (Popov et al., Reference Popov2010).
The logic model of the program is presented in Figure 1. It shows how the different components of the program are linked to its intended objectives and goals. The model suggests that positive changes in attitude and knowledge, as well as behavior, will lead to optimization of cognitive vitality. Cognitive vitality refers to the “development and preservation of the multidimensional cognitive structure that allows the older adult to maintain social connectedness, an ongoing sense of purpose, and the abilities to function independently, to permit functional recovery from illness or injury, and to cope with residual functional deficits” (Hendrie et al., Reference Hendrie2006, p. 13). It is presented here as a determinant of active aging given its considerable benefits to elderly people’s quality of life (Fillit et al., Reference Fillit2002).
Logic model for the Jog Your Mind program.
The logic model allowed deriving research objectives and hypotheses for the evaluation of the program. Indeed, after the pilot testing of the Jog Your Mind program, it was essential to evaluate its effectiveness through a larger quasi-experimental study, and to document the implementation settings and process to improve the program.
The main research objectives were: (1) to verify the effects of the Jog Your Mind program immediately and 12 months after the baseline; (2) to explore individual, organizational, and environmental factors associated with short-term and medium-term effects (e.g. baseline cognitive level); (3) to document the implementation settings and process of the program (e.g. population reached, adherence to the program), and to identify barriers and facilitating factors to implementation.
The pursuit of these three objectives will allow for verification of the following hypotheses: (1) immediately after the program, the program's participants will use more mnemonic strategies and will have improved subjective memory functioning in daily life compared to those who did not participate in the program. Other variables (practice of physical activity, participation in stimulating occupations, cognitive abilities, attitudes, and knowledge) will also be improved among the program's participants; (2) at medium term, i.e. 12 months after the baseline, these changes will be maintained; and (3) the short-term and medium-term effects of the program on the different variables will be weaker for people who are older, have a lower cognitive level, a lower socioeconomic status, a lower education level, are less active and who adhere less to the program (e.g. attendance of sessions, home practice). There will be fewer effects in community organizations located in high poverty index neighborhoods.
A quasi-experimental design was chosen for testing the hypotheses mentioned above. The trial was designed to mimic natural conditions as much as possible by giving community organizations the responsibility of recruiting participants and group leaders, as well as implementing the program. This paper describes the evaluation protocol of the Jog Your Mind program and the particularities of conducting research in partnership with multiple community organizations within natural settings.
Methods
Study design and rationale
The design was developed with the goal of achieving scientific rigor while maximizing broad generalizations of the results. As mentioned by some authors (Des Jarlais et al., Reference Des Jarlais, Lyles and Crepaz2004), results from controlled trials are not sufficient. Research must also produce knowledge on factors that can influence the effects of interventions when they are implemented in natural conditions. A quasi-experimental design with a one-year waiting list control group was chosen to evaluate the Jog Your Mind program and to document its implementation process. Because it promotes external validity, such design produces results that are particularly relevant to practitioners and decision-makers (Glasgow et al., Reference Glasgow, Lichtenstein and Marcus2003). This research design corresponds to an effects analysis of “use or practical efficacy” (Contandriopoulos et al., Reference Contandriopoulos, Champagne, Denis and Avargues2000), meaning that it aims to verify whether the program works under natural conditions. Randomization of individuals or groups was therefore excluded, in order to maximize acceptability of the research among community partners.
Figure 2 shows the overall design of the study. It involved two kinds of groups studied in parallel: the experimental and control groups. The experimental group participants took part in the program during the study. To limit differences between recruited participants as much as possible, the control group participants could take part in the program one year after their entry in the study. All participants were evaluated at three times during the study: (T1) at baseline (pre-program for the experimental group); (T2) 14 weeks later (post-program for the experimental group); and (T3) a 12-month follow-up from baseline. These three evaluation times allowed for the assessment of short-term and medium-term effects of the program. A 12-month follow-up from baseline was chosen to have measurements at the same period of the year for both the baseline and the last follow-up, as well as to see a one-year evolution among the control group. The study was planned over a four-year period (2009–2013) and recruitment and data collection were done in four phases (fall 2009, spring 2010, fall 2010, and winter 2011; Figure 3). Experimental and control groups were matched for the time of year that they entered the study to control for potential effects of the season on elderly people’s participation and lifestyle habits (e.g. level of physical activity).
Study design.
Planning of the study.
This protocol was approved by the research ethics committee of Université de Montréal. All community organizations received a copy of the ethics certificate. In addition, all coordinators of community organizations, group leaders, interviewers, and anyone involved in participant recruitment, program implementation, and data collection, signed a confidentiality form.
Intervention description and content
As mentioned previously, Jog Your Mind is a community-based program aiming to promote cognitive vitality in older adults without known or diagnosed cognitive impairment. It is offered to groups of seven to 15 elderly individuals over ten weekly sessions of 2 h with non-expert human resources, such as community practitioners or volunteers acting as group leaders. A group format was chosen as it allows for the use of strategies such as modeling, sharing of tips and resources, normalization, and networking. Hence, in addition to the acquired knowledge and the progress targeted, the program also promotes the development of social relationships.
The program's content has been described elsewhere (Popov et al., Reference Popov2010). Briefly, the program is comprised of cognitively stimulating activities, teaching of mnemonic tips, strategies promoting self-efficacy, initiation to an autonomous walking program, exploration of neighborhood resources, and promotion of healthy behaviors for maintaining cognitive functions. The logic model (Figure 1) shows the modalities associated with the program's strategies. Mnemonic strategies (e.g. attention focusing, categorization, use of memory aids) are practiced in a context that simulates daily tasks (e.g. remembering names, learning phone numbers). Intellectually stimulating activities (e.g. riddles, logic games, memorization tasks) are performed in each session. Some activities are also suggested as homework. The goal is to expose participants to a wide variety of leisure activities that are easily accessible to the public, in order to promote self-management of their cognitive stimulation. For each theme discussed, participants and the group leader explore the available community resources so that benefits of the program can be maintained and the pursuit of a personal goal may continue after the program. Participants are also invited to start an autonomous walking program. In that regard, the group leader uses cognitive-behavioral strategies to motivate the participants (e.g. pedometers, a walking log, personal objectives).
Implementation
Partner community organizations were in charge of implementing the program. For the study, group leaders were referred by the community organizations to participate in the one-day training offered by the program managers at CSSS Cavendish. After the training, leaders and community organizations benefited from support and advice offered by the CSSS Cavendish to assist with implementation. A detailed facilitation manual (Parisien et al., Reference Parisien, Gilbert, Amzallag, Guay, Nour and Laforest2013a; Reference Parisien, Gilbert, Amzallag, Guay, Nour and Laforest2013b) was offered to partner organizations. They also received compensation to cover costs inherent to their participation in the study.
Recruitment
Site recruitment
Twenty-three community organizations for elderly people were recruited in the metropolitan Montreal (Canada) area to be part of the study. The research coordinator used several strategies to recruit them: presentation of the project within community organizations, announcements in newsletters and invitations to organizations in the region. Thirteen of these community organizations were assigned to the experimental group, meaning that they would implement the Jog Your Mind program in their organization during the study. The other ten organizations were assigned to the control group, meaning that they agreed to wait at least 12 months (or after the study) before implementing the program within their organization. Assignment to the experimental or control group was mainly based on an organization's readiness to implement the program and on the needs of the study. No randomization was done so as not to manipulate the implementation process to the extent possible, and to respect the reality of the partner community organizations. All organizations signed an agreement with the research team and committed to offering the program at least once in order to be included in the study.
Participant recruitment
The responsibility of recruiting elderly participants was given to the coordinators of the partner community organizations. They were invited to use their usual strategies (e.g. newspapers ads, activity calendar, demonstrations of the program) and to use Jog Your Mind's promotional tools (e.g. pamphlets, posters) in order to recruit about 15 elderly individuals (to end up with 12 per group after screening). Study criteria were specified for the organizations in order to standardize the selection of participants. Both experimental and control groups were given the same instructions. In order to further ensure the comparability of the groups, control group participants were recruited on the condition that they participate in assessments for the study during the first year and that they would be invited to participate in the program the year after.
The participants were eligible to participate in the study if they met the following criteria: (1) aged 60 years old and over; (2) able to speak French; (3) accept to participate in a study over a one-year period; (4) interested in participating in a program promoting cognitive vitality over ten weekly 2-h sessions; (5) not having followed a similar program over the last year and; (6) not having received a cognition-related diagnosis by a healthcare professional (self-reported).
Group leader recruitment
Community organizations also had the responsibility of recruiting group leaders who would offer the program. For that purpose, they were invited to refer two members of their team. The only criterion specified for choosing group leaders was that these people had to demonstrate an interest and have previous experience in teaching. Group leaders could be elderly volunteers or community practitioners (e.g. special care counselors, psychosocial practitioners, recreation technicians). In addition to the one-day training offered by the research team, group leaders were given the opportunity to observe a program session before they began leading the program. It was suggested that they be paired together, especially when a group leader had limited previous experience in teaching.
Sample size and power calculation
The goal was to recruit 144 participants in total for the experimental group and 144 participants in total for the control group, for a total of 288 participants. The 25% attrition rate between the first and second evaluation as initially projected, would allow for 108 people per group, for a power of 80% for detecting medium-size effects following the program (α = 0.05) for one of the primary outcomes, namely the use of everyday memory strategies and aids. The power was expected to be somewhat weaker for medium-term effects (at the 12-month follow-up from baseline), given an additional projected attrition of 25% between the second and third evaluation. Other researchers have detected comparable effects with a similar variable evaluating the use of everyday memory strategies (Troyer, Reference Troyer2001).
Data collection
Once the recruitment period was over, the coordinator of each organization sent the list of names and contact information of potential participants to the research coordinator. A first phone contact allowed to re-verify participants’ eligibility and interest in participating in the study and program. People who did not meet the selection criteria were not included in the evaluation study. However, they were allowed to follow the program within their community organization. Participants were told in which group (experimental or control) they belonged and gave their verbal consent to participate in the study and to share socio-demographic data over the phone. A depression-screening test, the short version of the French Geriatric Depression Scale (Clement et al., Reference Clement, Nassif, Leger and Marchand1997), was also administered on the phone. If distress was detected on the phone or during the interviews, the participant was referred to a listening and referral center. Appointments for all three evaluations were booked during this first phone contact. Each participant was evaluated at three times during the study (T1, T2, T3) by the same interviewer, most of the time. Because elderly adults’ performance in cognitive testing can be influenced by fatigue, participants were tested at the same time of day for the three interviews.
At the beginning of the first interview, participants were asked to sign a consent form. They received a token amount of $10 per evaluation. Interviewers attended a two-day training offered by the research team (more details about the interviewers’ training are available in File S1, available as supplementary material attached to the electronic version of this paper at www.journals.cambridge.org/jid_IPG). They were not told in which group (experimental or control) the participant belonged in order to minimize the experimenter's bias, nor were they aware of the program details and objectives. Each interview lasted around an hour and a half. The outcome variables evaluated and the procedures at T1, T2, and T3 were the same.
Data on the settings and the implementation process were collected during the study. Community organization coordinators and group leaders were interviewed on the phone for a process follow-up at T2. Participants from the experimental group were interviewed both before and after the program to learn about their expectations and appreciation of the program.
Assessment tools
Outcome measures
Outcome measures and descriptions of related assessment tools are presented in Table 1. Keeping in mind the objectives and the community-based context of the study, the assessment tools were chosen based on the following criteria: (1) psychometric qualities (e.g. validity among the target population, sensitivity to change); (2) could be administered by trained interviewers who were non-experts in cognition; (3) length of time for administration allowing for an interview of less than 2 h; and (4) ability to measure the different dimensions of the logic model. In order to ensure the validity and reliability of these tools for the target population, a pre-test of the measurement tools was conducted among a representative sample of the target population before the study (n = 15). As presented in Table 1 (last column), some measurement tools were modified following this pre-test, to adapt them to the needs of the study. The pre-test also allowed for confirmation of the validity, reliability, and absence of ceiling effects of the chosen tools for our population. The main independent variable was the type of group (experimental or control) as an indicator of exposure to the program Jog Your Mind. Dependent variables were chosen according to their ability to measure the specific effects of the program or to measure cognitive vitality.
Outcome measures and measurement tools
Primary outcomes were the use of everyday memory strategies and aids, assessed by the Multifactorial Memory Questionnaire Strategy (MMQ-Strategy; Troyer and Rich, Reference Troyer and Rich2002; Fort et al., Reference Fort, Adoul, Holl, Kaddour and Gana2004), and subjective memory functioning in daily life, assessed by the Multifactorial Memory Questionnaire Ability (MMQ-Ability; Troyer and Rich, Reference Troyer and Rich2002; Fort et al., Reference Fort, Adoul, Holl, Kaddour and Gana2004). As secondary outcomes, the following dependent variables were evaluated: (1) attitudes related to cognitive functions, such as confidence (Metamemory in Adulthood Capacity (MIA-Capacity); Hultsch et al., Reference Hultsch, Hertzog and Dixon1987; Dixon et al., Reference Dixon, Hultsch and Hertzog1988) and feelings of control (Metamemory in Adulthood Locus (MIA-Locus); Dixon et al., Reference Dixon, Hultsch and Hertzog1988; Dellefield and McDougall, Reference Dellefield and McDougall1996); (2) knowledge about normal aging, resources, mnemonic strategies (homemade questionnaire plus an adapted version of the Memory Situation Questionnaire; Troyer, Reference Troyer2001); (3) adoption of behavior related to cognitive vitality, such as physical activity (12-item questionnaire inspired from the Community Healthy Activities Model Program for Seniors (CHAMPS) questionnaire; Stewart et al., Reference Stewart, Mills, King, Haskell, Gillis and Ritter2001) and participation in stimulating activities (18-item questionnaire adapted from the Florida Cognitive Activities Scale; Schinka et al., Reference Schinka, McBride, Vanderploeg, Tennyson, Borenstein and Mortimer2005); (4) cognitive performances in daily life situations (Rivermead Behavioural Memory Test; Vannier and Lemyse, Reference Vannier and Lemyse1994; Wilson et al., Reference Wilson, Cockburn and A.1994; Lezak, Reference Lezak1995); Attention in daily life (Coyette et al., Reference Coyette, Arno, Leclercq, Seron, Van der Linden and Grégoire1999); (5) cognitive abilities, more specifically memory and executive functions (Stroop test; Stroop, Reference Stroop1935; Golden, Reference Golden1978); California Verbal Learning Test II; Delis et al., Reference Delis, Kramer, Kaplan and Ober1987; Reference Delis, Kramer, Kaplan and Ober2000) and; (6) perception of memory on a scale from 1 to 10 (1 = very bad; 10 = excellent).
Variables related to the intervention process
Program delivery (e.g. number and length of sessions, group size, fidelity with manual content) was documented. Group leaders recorded attendance for every session so that adherence and attrition rates could be calculated. Implementation fidelity was evaluated during a phone interview with each group leader at the end of the program. Their satisfaction with the program and the one-day training session, as well as their level of facilitating experience, were also evaluated through this interview. A phone interview conducted after the program with the coordinator of each partner community organization (experimental and control groups), allowed us to collect organizational variables, such as the type and size of the organization. The perceived level of difficulty to implement the program and the willingness to offer it in the future, were also assessed in both the experimental and control groups. Barriers, facilitators and required resources for the program's implementation were also assessed during this interview for experimental groups. Participants’ expectations were gathered during phone interviews before the Jog Your Mind program, and their satisfaction and the perceived benefits were assessed after the program. Lastly, environmental variables (e.g. poverty index and level of urbanization) were collected from census data.
Control variables
Socio-demographic variables such as age, gender, marital status, living alone, years of education, perception of socio-economic status, and retired status were collected as control variables. Perceived physical health (Daveluy et al., Reference Daveluy, Audet, Courtemanche, Lapointe, Côté and Baulne2000; Daveluy et al., Reference Daveluy, Pica, Audet, Courtemanche, Lapointe and Côté2001), perceived mental health (Statistique Canada, 2005), health problems (Daveluy et al., Reference Daveluy, Audet, Courtemanche, Lapointe, Côté and Baulne2000), medication consumption (Daveluy et al., Reference Daveluy, Audet, Courtemanche, Lapointe, Côté and Baulne2000; Daveluy et al., Reference Daveluy, Pica, Audet, Courtemanche, Lapointe and Côté2001), perceived level of stress (Statistique Canada, 2005), life satisfaction (Statistique Canada, 2005), and sense of belonging to local community (Statistique Canada, 2005) were also collected. Participants were also asked to report vision and hearing levels (Santé Québec, 1998), alcohol and tobacco consumption (Statistique Canada, 2005), changes in diet (Daveluy et al., Reference Daveluy, Audet, Courtemanche, Lapointe, Côté and Baulne2000; Daveluy et al., Reference Daveluy, Pica, Audet, Courtemanche, Lapointe and Côté2001), and participation in other health promotion programs during the study. Control variables related to memory and cognitive level included: memory concerns reported on a scale from 1 (not concerned at all) to 10 (very concerned), general cognitive level assessed by the Montreal Cognitive Assessment – MoCA (Nasreddine et al., Reference Nasreddine, Chertkow, Phillips, Whitehead, Collin and Cummings2004) and whether the participant had discussed memory concerns with a health professional. Lastly, depression symptoms were evaluated using the short French version of the Geriatric Depression Scale (Clement et al., Reference Clement, Nassif, Leger and Marchand1997). All of this information allowed us to create a portrait of the participants. It was also used to compare characteristics of the participants in both the experimental and control groups and to identify confounding variables that were possible to control during the statistical analyses.
Analysis plan
Data entry and cleaning, and variable transformation were performed using the SPSS and Stata. The same research assistant cleaned all questionnaires before data entry. The scoring of some cognitive tests (e.g. the visuoconstructional skills task of the MoCA) was done by an occupational therapist, and borderline scores were verified by a neuropsychologist. Descriptive analyses were conducted to document characteristics of the population reached and certain variables related to the implementation context and process. Group differences at baseline were assessed through t-tests and χ 2 analyses. Any potentially confounding variables were taken into account in multivariate analyses.
The main analyses were comprised of simple comparisons. First, to compare groups at both T2 and T3, crude data were examined. The standardized effect was calculated in order to assess the sensitivity of the instrument for detecting a clinically significant difference (Kazis et al., Reference Kazis, Anderson and Meenan1989). Afterwards, a regression analysis controlled for potentially confounding variables at the individual level (e.g. age, socioeconomic status). Analyses were conducted according to the intent-to-treat principle and participants were maintained in the group to which they were initially assigned. Like in other studies, it was decided not to adjust for multiple comparisons (Rothman, Reference Rothman1990).
Lastly, exploratory analyses allowed us to observe how some individual variables (e.g. cognition level) and variables related to the implementation context (e.g. group leader's characteristics) are associated with effects of the program. Regression analyses, including interacting effects, were useful in documenting the differential effects of the program and identifying characteristics of participants who were most likely to benefit from this kind of intervention.
Proceedings of the study
As shown in Figure 2, 13 community organizations referred 171 potential participants for the experimental group and ten community organizations referred 202 potential participants for the control group. Screening for the study began in August 2009 and the last participant entered the study in January 2011. After the first phone contact with potential study participants, 28 and 51 participants were excluded from the experimental group and control group respectively, leaving 143 participants in the experimental group and 151 participants in the control group at T1 (baseline). Between T1 and T3, an attrition rate of 15.2% was obtained, which was lower than expected. Of the 294 participants recruited, 251 remained at T3, thus leaving the power of the study intact. Participants were lost due to health problems, traveling, their decision not to continue, the difficulty of the study, their belief that the program would not meet their needs or if it was impossible to contact them.
In total, the program was offered to 17 groups of older adults (three community organizations gave the program to two or three groups). Twenty-two group leaders were involved to deliver the program during the study. Group leaders were paired to co-facilitate the program for five experimental groups. All group leaders, except one, shared their appreciation of the program and their experience at the end of their mandate. Follow-up with each community organization was carried out after the end of the program. Each community organization coordinator who was responsible for a group provided his or her feedback on the implementation process and the program itself.
With respect to data collection, the majority of interviews were conducted in a room at a partner community organization as was intended. However, some of the interviews (23%) were conducted in the participants’ homes to accommodate them. In addition, for practical reasons (e.g. conflicting schedules or loss of an interviewer), some participants did not have the same interviewer for all three evaluations. In general, the evaluation procedures were well tolerated by participants.
Discussion
This paper described the protocol of a quasi-experimental study designed to evaluate a multifactorial health promotion program. This protocol will allow verifying whether participants to the Jog Your Mind program have better improvement in cognitive and health outcomes than participants of the control groups on short and medium term after the program. As the goal of the paper was to describe the protocol only, this section will discuss data related to the sample, the attrition rate, and the recruitment before addressing the methodological challenges associated with community-based research. Data related to the hypotheses and to the outcomes will be the object of future publications.
Participant recruitment and retention
Overall, recruitment and retention of participants were successful. The sampling goal was achieved with 294 participants being admitted in the study at baseline. This success may be explained by the fact that the Jog Your Mind program and study respond to an unmet need and that memory is a significant concern among elderly people. They thus sparked interest in a great number of people. In addition, community organizations were responsible for participant recruitment. As many participants had a certain sense of belonging to their neighborhood, this may have motivated them to participate in the study. Given the recruitment challenges usually faced during controlled trials such as refusals due to personal-, contextual-, or research-related factors (Gul and Ali, Reference Gul and Ali2010), taking advantage of community organizations’ relationships with the participants may be one way to overcome them. As for retention, we observed a low attrition rate of 15.2%, which was lower than expected. This was also an achievement, given that high attrition rate can reduce statistical power and limit internal and external validity. Our low attrition rate may be attributed to the fact that the same interviewer was assigned to a participant for all three evaluation times (T1, T2, and T3). This allowed for some continuity for the participants and enabled them to develop a relationship of trust with the interviewer. Furthermore, some interviews were conducted in the participant's home to avoid losing participants due to, for example, mobility issues. Hence, attention to the interviewer–participants relationship and flexibility in the procedures may have helped to keep participants in the study. A low attrition rate has also been obtained in other studies using comparable field procedures (Robitaille et al., Reference Robitaille2005). In our case, such good retention is also indicative that the Jog Your Mind study and the program sparked interest in a great number of people given that memory is a significant concern among elderly people and that the program responds to an unmet need in this regards.
Originality of the Jog Your Mind program
Compared to traditional cognitive training interventions (Tardif and Simard, Reference Tardif and Simard2011), the Jog Your Mind program distinguishes itself by its multifactorial approach. It combines cognitively stimulating activities, use of everyday memory strategies, self-efficacy promoting strategies, promotion of physical activity, awareness of neighborhood resources, and promotion of healthy behaviors for maintaining cognitive functions. According to Rebok and colleagues (Reference Rebok, Carlson and Langbaurn2007), this type of multifactorial approach is promising but has rarely been implemented. Jog Your Mind empowers elderly people to take charge of their health through its strategies, content, and activities. Its development and implementation, carried out in close partnership with community organizations and lay people acting as leaders, allow for enhanced program sustainability and community empowerment (Fienieg et al., Reference Fienieg, Nierkens, Tonkens, Plochg and Stronks2012). The program is simple enough to be administered by people with no specific clinical background, thus enhancing its potential for dissemination. However, the program does not include a booster session like other cognitive training programs (e.g. Willis et al. (Reference Willis2006)), which is a possible limitation. A booster session could help to maintain training-related benefits over time and facilitate the transfer of training to the activities of daily life required to maintain independent living (Rebok et al., Reference Rebok, Carlson and Langbaurn2007). It is thus possible that the results of this study show a weaker effect than other cognitive training interventions. In the same way, while its multifactorial approach is innovative, the Jog Your Mind program may not be as intensive and specific as other interventions, thus making it harder to detect large effect size.
The co-construction process used to develop Jog Your Mind (Nour et al., Reference Nour2010; Popov et al., Reference Popov2010) was a key preparatory step for conducting the evaluation study. This development process promotes the engagement of stakeholders from the beginning of the program's development. This allows for the creation of multiple partnerships with community organizations, which are also needed during the evaluation study. In addition, building a logic model during the development of the program allows for the identification of research objectives that are both relevant for the researchers and the community partners (Kaplan and Garrett, Reference Kaplan and Garrett2005). Lastly, testing of the program provides the opportunity for pre-testing and modifying the measurement instruments that are to be used during the evaluation study.
Methodological challenges
In line with the recommendation of many authors to conduct more pragmatic effectiveness trials (Glasgow et al., Reference Glasgow, Lichtenstein and Marcus2003), efforts were made to mimic natural conditions as much as possible when designing the Jog Your Mind evaluative protocol. One major aspect of the protocol reflecting this decision was the fact that community organizations were made responsible for the recruitment of participants and group leaders and for the program delivery. This had various implications on the design as described below.
First, using this strategy did not allow for randomization, since many of the community organizations may not have been ready to implement the program. The research team had to opt for alternative designs. While some may see these alternative designs as less effective in eliminating potential confounding variables, a good understanding, rigorous measuring and control of these variables can make evidence from these trials more convincing (Bonell et al., Reference Bonell2011). In addition, it is worth noting that randomization does not, in and of itself, promote external validity, which is usually the main concern in these types of quasi-experimental pragmatic trials (Glasgow et al., Reference Glasgow, Lichtenstein and Marcus2003; Bonell et al., Reference Bonell2011). Given that little is known about cognitive intervention effectiveness in natural conditions, applicability of the study's results to community settings was emphasized.
Second, there may be less control available on the research process and program delivery, which may increase the possibility that differences between settings or between group leaders influence the study's outcomes. As mentioned by Allison and Rootman (Reference Allison and Rootman1996), while sharing control among multiple community partners is “consistent with the philosophical goals of health promotion research, it does have implications for scientific rigor in the traditional sense of the term” (p. 338). To minimize selection bias, a clear description of the participant selection criteria was provided to all community organizations. The same guidelines were given to both the experimental and control groups. Furthermore, to ensure a standardized implementation of the Jog Your Mind program across all sites, specific measures were taken. For instance, all group leaders received a 7-h training about the program and its application. They were asked to follow the guidelines of the Leader's Manual that was provided to them. The research team also provided counseling and advice to group leaders and to the person in charge of implementing the program in each organization when needed. Lastly, a visit was made to the organization before the program to ensure that the setting was adequate. Unfortunately, program sessions were not observed for practical reasons (e.g. lack of time). Like in other studies (Filiatrault et al., Reference Filiatrault2007), observation of one or two sessions during the program could have allowed for an objective assessment of the community organizations and compliance of the group leaders to the program guidelines.
While the primary concern in designing this study protocol was the applicability of results to community settings, internal validity of the protocol was also given consideration through careful selection of measurement tools with sound psychometric properties and rigorous processes to limit attrition, interviewer bias, and contamination.
Measurement tools were carefully selected for their psychometric properties. In this regard, the pre-test of processes and data collection tools conducted before the study, was very useful. In fact, given that the older population without known or diagnosed health conditions is not often studied, the pre-test allowed for confirmation of the choice of measurement tools as a function of their variability and the absence of a ceiling effect. In addition, it allowed for verification of the feasibility of data collection processes and adjustment of its practical aspects (e.g. readability of documents, administration time). Interviewer bias was also given some consideration. In fact, the research team did not inform interviewers as to the group (experimental or control) in which participants were being evaluated. Participants were also reminded not to offer this information to the interviewers. Training and coaching of interviewers provided during the study, as well as a detailed interviewer's manual prepared by the research team, certainly helped to maintain rigor and standardization of the data collection.
Conducting research within the community has the primary advantage of reducing the gap between research and practice (Glasgow et al., Reference Glasgow, Lichtenstein and Marcus2003). In fact, research findings from pragmatic trials are particularly relevant to practitioners and decision-makers given that they have been tested under natural conditions and they promote external validity (Glasgow et al., Reference Glasgow, Lichtenstein and Marcus2003). For the Jog Your Mind study, a wide variety of community settings (e.g. community organizations, elderly people’s residences) and non-expert group leaders were encouraged in order to maximize the applicability of the results for community practitioners. Many authors advocated for researchers to conduct more research under natural conditions so that knowledge could effectively be translated into practice (Glasgow et al., Reference Glasgow, Lichtenstein and Marcus2003; Bonell et al., Reference Bonell2011). Many researchers have yet to realize that the additional time, patience, and negotiation with community partners required for community-based research, do not necessarily lead to a loss of rigor (Allison and Rootman, Reference Allison and Rootman1996).
Conclusion
This paper described the evaluation protocol of the Jog Your Mind program and the particularities of conducting research in partnership with multiple community organizations within natural settings. As few multifactorial community-based programs promoting cognitive vitality among elderly people have been documented, the results of this study will fill an important gap in the literature and will provide new knowledge on important questions concerning the effectiveness of such a program. This knowledge will help guide decision-makers in planning activities for elderly people that promote cognitive vitality and will also have implications for community organization practitioners who are looking for a creative cognitive vitality program.
From a methodological perspective, the Jog Your Mind protocol can serve as an example to guide future research aiming to evaluate health interventions under natural conditions. Indeed, it describes a number of methodological considerations allowing researchers conducting community-based studies to achieve scientific rigor while being sensitive to the reality of community settings. Those considerations include pre-testing and selecting measurement tools with sound psychometric properties, giving the community organizations responsibility for participant recruitment and program implementation, providing specific training to interviewers in order to ensure the standardization of the study protocol and intervention implementation. It is hoped that this protocol will inspire other researchers, not only to conduct more research in natural conditions, but to do so with rigor.
Conflict of interest
This research was funded by the Canadian Institutes of Health Research (grant #102 545). This sponsor played no role in the research other than funding the research.
Description of authors’ roles
Nathalie Bier, PhD, is a co-researcher for this study. She contributed in formulating the research questions and designing the study. She was also responsible for managing the preparation of the paper. Agathe Lorthios-Guilledroit, MSc, assisted with the literature review and the writing of the paper. Kareen Nour, PhD, Manon Parisien, MSc, and Dave Ellemberg, PhD, are also co-researchers for this study. They contributed in formulating the research questions, designing the study, and writing the paper. Sophie Laforest, PhD, is the principal investigator for this study. She too contributed in formulating the research questions, designing the study, and writing the paper.
Acknowledgments
The authors wish to thank the community organizations and the elderly individuals who participated in this project. They are also grateful to Danielle Guay who was the research assistant for this study.
Supplementary materials
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1041610214001306
