Introduction
Positive psychiatry offers an unique approach to promote brain health and well-being in aging populations (Jeste and Palmer, Reference Jeste and Palmer2015). Rather than focusing solely on disease management and treatment after illness occurs, positive psychiatry seeks to disrupt the “disease paradigm” of medicine, using methods that contribute to mental health and well-being, resilience, and lower risk of developing illness, including dementia (Cassidy and Cassidy, Reference Cassidy and Cassidy2019). Preventative approaches using positive psychiatry principles also stand to reduce burden on healthcare resources and healthcare costs (Dudgeon, Reference Dudgeon2010). Yet, clinicians often lack confidence, tools, and training in behavioral activation techniques (Keyworth et al., Reference Keyworth, Epton, Godlthorpe, Calam and Armitage2019). The Fountain of Health (FoH) (www.fountainofhealth.ca) is an “applied” positive psychiatry initiative serving to close this knowledge-to-practice gap in frontline care in Canada. The FoH is cited as an example of “positive outcomes-oriented research (with)…potential to significantly improve the lives of older adults” (Stoner, Reference Stoner2019). The FoH provides clinicians with the confidence, tools, and training to use a systematic evidence-based approach to patient behavior change with reliable outcome measures for well-being, health and resilience, and attitudes and behaviors (Gough et al., Reference Gough, Cassidy, Rabheru, Conn, Canales and Cassidy2018; Thoo et al., Reference Thoo, Freer and Cassidy2015). The FoH approach uses specific and measurable outcomes such as goal attainment following goal setting to capture the degree to which participants implement behavior change, showing that knowledge transfer (KT) interventions “can be an effective means of promoting a healthy lifestyle and improve well-being” (Stoner, Reference Stoner2019). In FoH KT, clinicians learn to help patients engage in behavioral activation using SMART (specific, measurable, action-oriented, realistic, and time-limited) goal setting (Doran, Reference Doran1981). SMART goals are set in five evidence-based modifiable protective behaviors to promote brain health including: physical activity, social activity, brain challenge, positive thinking, and taking care of mental health (Cassidy and Cassidy, Reference Cassidy and Cassidy2019; Livingston et al., Reference Livingston2017). A recent call has been made to increase rigor in positive psychiatry methodology (Stoner, Reference Stoner2019). Relevant to positive psychiatry interventions, despite the demonstrated value of goal setting for effective behavior activation (Locke and Lathan, Reference Poppes, Vlaskamp, De Geeter and Nakken2002), a standardized method for quality goal setting is currently lacking (Poppes et al., Reference Poppes, Vlaskamp, De Geeter and Nakken2002). Healthcare is increasingly reliant on remote care strategies such as health apps to support patient well-being, yet little is known about the differences in user profiles and patient outcomes between clinician-assisted versus self-guided app use for behavioral activation.
The current study was conducted as The Brain Health and Wellness Project, a national KT project, led by the Canadian Coalition of Seniors’ Mental Health (www.ccsmh.ca). This project was a multi-site quality assurance (QA) project providing KT to over 2200 clinicians across Canada in 2019 using FoH tools to facilitate individual patient behavior change to promote brain health using two methodologies: (1) a clinician-assisted paper-based format and (2) an electronic web-based app as an independent self-management tool that could be used with or without clinician support (The Wellness App: www.wellnessapp.ca). Both formats were minimal intervention methodologies that could be completed in a few minutes across two patient contacts. Patients were invited to set a baseline of health beliefs and attitudes, and to write one SMART health goal, in one of the five above-mentioned health domains. About 1-month later, patients were invited to complete patient reported outcomes (PROs) on goal attainment, well-being, and health attitudes. The overall effectiveness results of The Brain Health and Wellness Project showed that regardless of methodology, paper-based tools or The Wellness App tool, the vast majority of patients improved health behaviors and health attitudes by at least partially attaining their health goal within 1 month, and reported significant improvements in well-being and health attitudes. Detailed reporting of KT methods, feasibility, and effectiveness results of this project are reported elsewhere (Rabheru et al., Reference Rabheru2020).
Behavioral activation positive psychiatry interventions to promote brain health and well-being are increasingly becoming available using self-guided apps. Little is known, however, about the effectiveness of app technology, or the difference between clinician-assisted versus self-guided app methodology for behavioral activation to support brain health and wellness. Literature exploring whether and how health apps could contribute to primary prevention and positive psychiatry is lacking. Additionally, it is unknown whether health apps can serve to augment, or possibly even replace, patient–clinician interaction to support behavioral activation and well-being. Therefore, the present study assessed the differences in user profiles and outcomes between two FoH positive psychiatry behavioral activation methodologies used within The Brain Health and Wellness Project: (1) clinician-assisted and (2) independent app use for self-management.
Study questions
To enhance rigor in positive psychiatry methodology (Stoner, Reference Stoner2019) by further refining useful outcomes measure like SMART goal setting and attainment, and to address the need for investigation of the role of health apps in positive psychiatry, the present study investigates differences in users and outcomes between two methods of the FoH interventions by assessing: (1) clinician-assisted and (2) independent app use for self-management. The present study answers the following questions: (1) Are clinician-assisted patients different from independent app users? If so, what were the different characteristics of these patient groups? and (2) Does the method of behavioral activation influence: (a) intervention processes (completion rate and SMART goal setting and adherence to key parameters outlined in SMART goal setting which will be referred to here on in as goal SMART-ness) and (b) psychological and behavioral patient health outcomes? If so, what are the differences?
Methods
Study design, data collection, and sample
Two methods of a behavioral activation intervention were retrospectively compared: (a) clinician-assisted paper-based clinical tools and (b) independent self-management app use (The Wellness App).
Participating clinicians were educated on how to invite health behavior change and support patients to set one SMART health goal. As a QA project, no exclusion criteria were applied; however, clinicians were encouraged to invite patients who were (a) over 40 years of age and (b) without a dementia diagnosis. While the FoH intervention has the potential to benefit those with mild cognitive impairment (Stoner, Reference Stoner2019), clinicians were instructed to only invite patients without cognitive impairment to increase the homogeneity in the patient sample. Data collection occurred between January and September 2019.
For the purpose of this analysis, two subsets of patients were selected for comparison: first, those who participated in the intervention with the support of a clinician using the paper-based clinical tools and second, those who used The Wellness App independently as a self-management tool. Patients who used The Wellness App may have been recommended to the app by a clinician but completed the 4-week intervention independently (without clinician assistance). Finally, post hoc analyses were conducted on a third group of participants, those who completed the intervention with the support of a clinician while also using The Wellness App as a self-management tool between clinical visits. The sample included 254 clinician-assisted patients, 333 independent app users, and 45 clinician-assisted app users.
Clinician-assisted behavioral activation intervention
The FoH behavior change minimal intervention occurred over the course of two visits, approximately 4 weeks apart. At Visit One, clinicians established a baseline of health attitudes and behaviors using the Health and Resilience Questionnaire and collaborated with the patient to set one SMART goal. Results on individual items of the questionnaire served as a starting point to identify health goal domains. For example, a patient who scored low in the physical activity subscale would be encouraged to set a goal in this domain. At Visit Two, clinicians followed up with their patient and administered PRO on goal attainment, well-being, and health attitudes. Clinicians were instructed to return all anonymous patient forms back to the project team at the end of the project.
Independent app use (The Wellness App) behavioral activation intervention
Upon app account creation, users provided demographic information (age, gender). Similar to clinician-assisted patients, independent app users were prompted to establish a baseline of health attitudes and beliefs using the electronic version of Health and Resilience Questionnaire, and to set one SMART goal. Four weeks after goal setting, independent app users completed PRO on goal attainment, well-being, and health beliefs and attitudes. Additional features of The Wellness App included audio encouragements to support goal attainment, web-based resources, and email reminders to complete daily electronic check-ins. Patients chose the frequency of email reminders (daily, twice a week, weekly, or none). Email reminders also prompted independent app users to complete the 4-week follow-up outcome measures.
Outcome measures
Psychological measures
Two psychological measures were obtained: the Health and Resilience Questionnaire and the PROs of change in well-being post-intervention.
At baseline, and 4 weeks later, patients completed the Health and Resilience Questionnaire, composed of 5, 10-point, visual analog scales assessing health beliefs and attitudes on key health domains used in positive psychiatry and known to be protective brain health factors (Cassidy and Cassidy, Reference Cassidy and Cassidy2019; Livingston et al., Reference Livingston2017). Specific ratings on social connection (not at all connected to very connected), interest in learning new things (not at all interested to very interested), self-defined successful aging (least successful to most successful), seeking mental health help if needed (not at all likely to very likely), and level of physical activity (not at all active to very active). Visual analog ratings were used to avoid the challenges of ordinal-level data associated with Likert scales (Sung and Wu, Reference Sung and Wu2018; Voutilainen et al., Reference Voutilainen, Saaranen and Sormunen2016). Total scores were calculated and used in the analyses. The use of visual analog scales for within-subject comparisons has been previously validated using classical test theory methods (Delgado et al., Reference Delgado2018), and the specific submeasure on self-rated successful aging has been validated as a positive psychiatry measure (Depp and Jeste, Reference Depp and Jeste2006).
At the 4-week follow-up, patients reported the extent to which they met their goals, and whether working toward their goal impacted their well-being. Goal attainment was measured using validated goal attainment scaling (GAS) (Krasny-Pacini et al., Reference Krasny-Pacini, Hiebel, Pauly, Godon and Chevignard2013; Rockwood et al., Reference Rockwood, Joyce and Stolee1997), in which participants chose among the following categories: no change, partially achieved, fully achieved, and exceeded. GAS guidelines recommend setting at least three goals, although recent research suggests GAS is also responsive to a single goal (McGarrigle et al., Reference Cassidy and Cassidy2019) as used in the present study. Subjective change in well-being related to working toward the goal was measured using a similar ordinal scale in which participants chose among the following categories: no change, slightly improved, moderately improved, and greatly improved. Categorical scores were analyzed using Chi-square analyses.
Behavioral measures
Two behavioral measures were obtained: (1) patient self-reported goal attainment using GAS, as described above obtained at the 4-week follow-up and (2) a calculated goal SMART-ness. In the present study, goal SMART-ness – or adherence to parameters outlined in SMART goal setting – was measured by the extent to which stated patient goals included SMART goal parameters. In the absence of a standardized SMART goal measure, the authors identified eight items to measure effectiveness of goal setting based on the literature and aspects of goal planning. The goal SMART-ness criteria used were (1) an identified health domain; (2) an identified location of the action; (3) a specified time of day of the action; (4) a specified duration of the action; (5) an identified frequency of the action; (6) the inclusion of problem-solving solutions to possible obstacles; (7) the identification of possible supports to achieve the behavioral goal; and (8) use of tracking/monitoring strategies. The sum of the criteria was calculated as the degree of SMART-ness of the goal for a maximum score of eight. This scale was developed by the authors in the absence of an established measure of this construct, and to enhance the utility of the scale within behavior activation to support positive psychiatry.
Analysis
Pearson’s Chi-square tests were conducted on nonordinal categorical variables, including gender. For ordinal variables (goal attainment and well-being) nonparametric Mann–Whitney U statistics were calculated. Significance level was set at p < 0.05 for these analyses.
Independent samples t-tests were conducted to assess mean differences between clinician-assisted and independent app methods based on a baseline for the individual Health and Resilience Questionnaire domains and total scores. Pre–post Health and Resilience items and total scores were analyzed using two-factor analyses of variance (ANOVA), with one between-subjects (clinician-assisted vs. independent app use) and one within-subjects (pre–post) factor. This analysis yielded three outcomes: clinician-assisted versus independent app use overall differences (independent of time); pre–post differences (independent of clinician-assisted vs. app); and a clinician-assisted/independent app use by pre–post interaction, which addresses the issue of whether the pre–post change in scores differ between clinician-assisted and independent app users. Significance level was set at p < 0.05 for these analyses.
Post hoc analyses were conducted for the third group of patients, clinician-supported app users. In this intervention method, patients use The Wellness App between visits as a tool for self-management in addition to clinician support for goal setting and clinician follow-up. All statistical analyses were conducted using SPSS (Version 25) software.
Ethics
As a multi-site QA study, The Brain Health and Wellness Project received exemption from Nova Scotia Health Authority, Ottawa Health Sciences Network, Baycrest Health Sciences, and the University of British Columbia. No personal identifying data were collected on patients.
Results
User profiles
A total of 587 patients met the criteria for analysis. Of these, 254 were clinician-assisted patients and 333 were independent app users. Clinician-assisted tool users and Wellness App users differed on two characteristics. On average, clinician-assisted tool users were (a) older (69.06 [SD = 11.05] years vs. 50.62 [SD = 13.61] years, t = 17.54, p < 0.001) and (b) had a higher percentage of males (57% vs. 43%); (χ 2 (2) = 15.995, p < 0.001). The percentage of patients who completed the 4-week follow-up assessment were compared. Striking differences resulted between methods, in which 99.2% of the clinician-assisted patients completed a 4-week assessment compared to 10.8% of independent app users. It is noteworthy that only 60.1% of independent app users (N = 202) set a health goal. In other words, only 73% of those who started a patient profile on The Wellness App set a SMART goal in the app, and 27% of users created a profile but did not set a SMART goal. Only participants who completed the intervention were included in the following statistical analyses.
Psychological outcomes
Health and Resilience
Differences were observed between groups. On average, mean scores at baseline on three of five individual health and resilience items varied (see Figure 1). On average, clinician-assisted patients scored lower than independent app users in the domains of social activity (M = 6.12 [SD = 2.45] vs. M = 6.59 [SD = 2.14], p < 0.05), lifelong learning (M = 6.93 [SD = 2.18] vs. M = 7.98 [SD = 1.78], p < 0.05), and total score (M = 32.63 [SD = 7.84] vs. M = 33.98 [SD = 6.95], p < 0.05). Clinician-assisted patients scored higher than independent app users on the item seeking mental health help (M = 7.87 [SD = 2.32] vs. M = 7.04 [SD = 2.34], p < 0.001).
Baseline scores on the Health and Resilience Questionnaire between those using the tools with clinician assistance and those using the app on their own (self-management). The figure demonstrates observed differences between groups on baseline scores of the Health and Resilience Questionnaire. On average, groups mean scores at baseline varied in three individual health and resilience items (social activity, lifelong learning, and seeking mental health help). Overall scores on this measure also varied between groups.
Pre–post scores were compared between those in the clinician-assisted and independent app user groups using ANOVA. These results are shown in Figure 2. For the total score, there was a significant increase from pre- to post-intervention, with no interaction effect. The same pre–post significant increase occurred for the social connection item, as well as the items assessing successful aging, seeking mental health help when needed, and being physically active. There was no significant overall difference between clinician-assisted and independent app groups who completed the intervention (recall that only 10.8% of independent app group completed the intervention, compared to 99.2% of those in the clinician-assisted group). On the item assessing interest in learning new things, no pre–post difference and no interaction effects were found. However, overall, independent app users scored higher than those in the clinician-assisted group.
Impact of clinician-led versus app-based behavioral activation on the Health and Resilience Total and Subscale Scores. The figure compares pre–post scores between those in the clinician-assisted and independent app user groups using ANOVA. Results of this figure demonstrate that for the total score, there was a significant increase from pre- to post-intervention, with no interaction effect. The same pre–post significant increase occurred for individual health and resilience items (social connection item, successful aging, seeking mental health help, and being physical activity). No significant overall difference between patient groups was found. On the item assessing interest in learning new things, no pre–post difference and no interaction effects were found.
Well-Being
With regard to well-being, 17% of clinician-assisted patients and 16.1% of independent app users reported no change in well-being, 33.1% of clinician-assisted tool users and 38.7% of independent app users reported slight improvement, 37.5% of clinician-assisted tool users and 35.5% of independent app users reported moderate improvement, and 11.7% of clinician-assisted tool users and 9.7% of independent app users reported great improvement. As these were ordinal data, a nonparametric test was used to compare the distribution across groups. The statistical test was selected by SPSS to best fit the data; in this case, the Mann–Whitney U test. Impact on well-being was not different between clinician-assisted and independent app use (Mann–Whitney p = 0.77).
Behavioral Outcomes
Goal attainment
With regard to goal attainment,: 11.5% of clinician-assisted patients and 10.8% of independent app users reported no change in goal attainment, 41.9% of clinician-assisted patients and 62.2% of independent app users reported partial goal achievement, 31.2% of clinician-assisted patients and 21.6% of independent app users reported full goal achievement, and 15.4% of clinician-assisted patients and 5.4% of independent app users reported exceeding their goal. No significant difference was found between clinician-assisted users’ and app users goal attainment (p = 0.051).
Goal SMART-ness
The degree of goal SMART-ness was the score of the quality of patient goals defined by the degree to which goals met specific parameters. SMART-ness criteria were the presence or absence of qualities of: (1) an identified health domain; (2) an identified location of the behavior; (3) a specified time of day of the action; (4) a specified duration of the action; (5) an identified frequency of the action; (6) the inclusion of problem-solving solutions to possible obstacles; (7) the identification of possible supports to achieve the behavioral goal; and (8) use of tracking/monitoring strategies. The sum of the above criteria was calculated to reflect the degree of SMART-ness of the goal, with a maximum goal SMART-ness score of eight. SMART-ness scores (representing ratio data) were then calculated on all individuals with a goal identified which was 99.2% clinician-assisted patients (N = 330) and 60.1% of independent app users (N = 202). Physical activity goals were strongly preferred by both user groups (55.2% of clinician-assisted patients and 60.1% of independent app users) over the other four FoH health domains (20% or less of participants selected any other domain). Goal domain selection differed between clinician-assisted patients and independent app users (see Figure 3) (χ 2 (4) = 10.76, p = 0.029), where more independent app users selected physical activity and positive thinking goals, and more clinician-assisted tool users selected goals related to social activity and mental health.
Percentage of patients selecting the different goal domains in their behavioral activation plan. The figure illustrates and compares the different goal domains chosen by patients for their behavioral activation plans between patient groups. More independent app users selected physical activity and positive thinking goals, and more clinician-assisted tool users selected goals related to social activity and mental health.
On average, clinician-assisted patients had a higher goal SMART-ness mean (M = 2.78 [SD = 1.08]) than independent app users (M = 2.44 [SD = 0.92], t = 3.54 p < 0.001).
A significant association was found between the method of intervention delivery and whether or not patients included details that would facilitate calculation of goal SMART-ness as part of their health goal. Clinician-assisted patients were more likely than independent app users to include goal items of: (a) Location (35.5% vs. 7.9%) (χ 2 (1) = 47.66, p = 0.001); (b) Time of day (11.6% vs. 5.4.%) (χ 2 (1) = 5.19, p = 0.023); (c) Problem-solving (4% vs. 0.5%) (χ 2 (1) = 5.75, p = 0.016); and (d) Support (8% vs. 1.5%) (χ 2 (1) = 9.76, p = 0.002). Independent app users were more likely than clinician-assisted patients to include the item of monitoring (0% vs. 4%) (χ 2 (1) = 10.12, p = 0.001). Clinician-assisted patients and independent app users did not differ in their likelihood of including the items of duration (42.4% vs. 45.5%) (χ 2 (1) = 4.99, p = 0.480) or frequency (82.1% vs. 79.7%) (χ 2 (1) =4.08, p = 0.523).
Combined paper tool and app users
As noted in the methods section, a smaller number of participants used The Wellness App as an adjunct to the clinician-assisted tool use. We analyzed the data from this group (N = 45). Examination of the results revealed that combined clinician-assisted app users scored more similarly to the clinician-assisted patients than the independent app users on most variables, such as age (mean = 63.8 years), gender (64.4% female), and completion rates (100%). On the Health and Resilience Questionnaire, users of both methods (clinician-assisted and independent app) scored high pre-measures (M = 36.4 [SD = 7.25]) and post-measures (M = 39.9 [SD = 6.31]). There was no group X time interaction (F = 1.37, ns) but a strong increase from pre to post for all groups (F = 38.71, p < 0.001). Regarding improvements in well-being, nonparametric statistics failed to show differences between the app use groups (Mann–Whitney U p = 0.12). However, on goal attainment, more of the users of both clinician-assisted and independent app reported higher levels of goal attainment (Mann–Whitney U p < 0.05). In the group using both methods, 6.5% reported no goal change, 31.1% reported partial attainment, 34.8% reported full attainment, and 19.6% reported exceeding their goal.
Discussion
As part of The Brain Health and Wellness Project, the present study assessed the differences in users and outcomes between two methods of delivery of the FoH positive psychiatry intervention: (1) clinician-assisted and (2) independent app use for self-management. Results suggest user differences in baseline characteristics. No differences were found between groups for post-intervention goal attainment or psychological outcome measures.
In the present study, compared to app users, patients assisted by clinicians were more likely to be male, older, and scored lower in individual domains of health and resilience at baseline. These latter findings echo previous research which found that health app users typically report sociodemographic factors that are associated with better health outcomes (e.g., younger, more educated, healthier, and higher income) (Carroll et al., Reference Carroll, Moorhead, Bond, Leblanc, Petrella and Fiscella2017). This finding suggests that health promotion interventions that rely solely on app technology may be less likely to engage individuals who could benefit the most. Clinicians are often the first point of contact for individuals experiencing first signs of dementia. For this reason, they are ideally positioned to educate on dementia lifestyle risk factors and to implement risk reduction interventions. Specifically, within the context of dementia risk reduction, brain health promotion interventions that include clinician support may be better suited to reach at-risk populations.
Health apps are gaining popularity for their potential to improve delivery of care, health outcomes, and patient support between clinical visits. Relevant to positive psychiatry, results suggest that no significant different was found between those who completed the FoH intervention independently using the Wellness App and those who worked with a clinician. An unanticipated benefit to app users was an increase in new learning, a self-reported health and resilience domain of interest, relative to those who did not use the app. Lifelong learning is a modifiable behavior known to improve long-term brain health (Valenzuela and Sachdev, Reference Valenzuela and Sachdev2009). It is possible that patients who are less familiar with technology might benefit from engaging with health apps as a novel brain activity to promote new learning.
While the benefits of app use are promising, the difference in completion rates between app users and those working with clinicians was striking. Only 10.8% of app users completed the intervention demonstrating the important role of interpersonal relationship for successful behavioral activation. Patient–clinician relationships are a known positive factor of treatment completion. A systematic review exploring treatment adherence in mental healthcare by Thompson and McCabe (Reference Thompson and McCabe2012) found patient–clinician alliance, which is the level of agreement on treatment goals and plan for goal achievement, as well as a bond between patient and clinician, were associated with greater treatment adherence. Patient–clinician relationships appear to be vital for effective health behavior change. Therefore, consideration is needed when recommending health apps alone for behavioral activation, and further research is required to explore how best to increase app user engagement and ultimately intervention completion. For example, app user friendliness could ensure a more seamless user experience, optimize retention, and maximize user success. Further investigation is needed to identify the app features that are best suited for the intended users. Interdisciplinary collaboration, between researchers and app developers, could enhance the design of health apps for increased uptake, engagement, and retention.
Multicomponent interventions result in greater health improvements (Schoeppe et al., Reference Schoeppe2016). Preliminary results of a small sample of clinician-assisted app users were promising. Continued exploration on the added value of health apps as an adjunct to clinician-led intervention is needed. Despite having found no significant difference between groups for goal attainment, in this study, the role of the clinician contributed to intervention completion and was associated with higher scores of goal SMART-ness. Clinician KT, as part of this project, emphasized how to set SMART goals and this could explain the association between the clinician-assisted method and increased goal SMART-ness. Not all patients who have the intention to adopt, maintain, or increase health behaviors will subsequently do so. This is known as the intention–action gap (Rhodes and de Bruijn, Reference Rhodes and de Bruijn2013; Sheeran and Webb, Reference Sheeran and Webb2016). Previous research has found that clinicians play a crucial role in supporting behavioral activation through strategies, such as helping to identify small steps to initiate action, demonstrating interest and support, as well as providing follow-up (Greene et al., Reference Greene, Hibbard, Alvarez and Overton2016), elements included in the clinician-assisted method.
Given the clinician-assisted methodology was associated with increased goal SMART-ness and approached significance for goal attainment (p = 0.051), it could be hypothesized that setting “SMART-er” goals might serve to increase the likelihood of goal attainment. Goal SMART-ness results show that the majority of patients, both independent app users and clinician-assisted, included the items frequency and duration while a minority explicitly explored other aspects of SMART-ness (e.g., location, time of day). We speculate that further increasing the degree of SMART-ness might lead to greater goal attainment and a significant relationship between degree of goal SMART-ness and goal attainment. Continued exploration into this relationship and elucidation of methodologies to further enhance goal attainment are needed. Identifying the relative importance or “weight” of goal SMART-ness items for goal attainment might also lead to greater goal success. For example, goals that include aspects of problem-solving might be more likely to lead to goal success than goals that do not, while goals that identify monitoring might be just as a likely to lead to goal attainment than goals that do not. Future investigation to explore this method and the influence of individual goal SMART-ness parameters on goal attainment is warranted.
In the absence of an established systematic goal-setting approach for behavioral activation, the SMART-ness parameters, outlined as part of this study, offer clinicians a method for writing and evaluating health goals, one that includes elements of goal planning such as problem-solving, monitoring, and support. Further, strategies to improve clinician guidance of goal writing for health promotion in frontline care are warranted. Exploring the effectiveness of educational strategies and materials could inform tailored knowledge dissemination approaches.
Strengths and limitations
The findings of this study should be considered in light of the following strengths and limitations. As far as we are aware, The Brain Health and Wellness Project is the first nationally scaled positive psychiatry initiative in the world and offers a significant contribution to improving clinicians’ access to tools and training to promote brain health and well-being. Considering the importance of behavioral activation to positive psychiatry as an intervention to promote health and well-being, the degree of rigor in outcomes measure like SMART goal setting and attainment have been lacking. This is the first study of which we are aware to enhance positive psychiatry research methodology by providing a preliminary measure of goal SMART-ness. There is a need to continue to explore methodologies to further enhance goal SMART-ness setting between patients and clinicians, and ultimately the attainment of goals that serve to promote patient well-being.
In terms of the novelty of this contribution, this is also the first positive psychiatry initiative of which we are aware to investigate the difference in users and outcomes between (1) clinician-assisted and (2) independent app use for self-management to promote health behaviors and improve well-being. The new findings are that the method of behavioral activation did influence (a) the intervention processes (completion rate and goal SMART-ness) but not (b) behavioral patient outcomes (goal attainment) and (c) the psychological patient health outcomes (health and resilience and well-being).
In terms of methodological strengths within positive psychiatry, the use of a visual analog scale for within-subject comparisons is a previously validated classical test theory method (Delgado et al., Reference Delgado2018); the specific submeasure on self-rated successful aging is a validated positive psychiatry measure (Depp and Jeste, Reference Depp and Jeste2006) and goal attainment is measured using the validated GAS (Krasny-Pacini et al., Reference Krasny-Pacini, Hiebel, Pauly, Godon and Chevignard2013; Rockwood et al., Reference Rockwood, Joyce and Stolee1997). The addition of a SMART-ness measure offers new opportunities for investigations into the relationship between goal SMART-ness and goal attainment as a methodology for behavioral activation within positive psychiatry. This study investigated whether goals that included more goal SMART-ness items led to greater goal attainment, and further investigations should consider the role of individual goal SMART-ness items for goal attainment.
A methodological limitation of this study is the lack of control over Type I error. However, given that a number of the statistical results were at the <0.001 level, the odds of reporting a false positive are minimal.
Of note, this is a retrospective analysis of a national QA initiative. The data analyzed came from a population of patients deemed by their clinicians appropriate for the behavioral activation intervention; therefore, a selection bias may have influenced intervention outcomes. Due to outcome measures used, self-report bias should also be considered.
Another limitation is that we did not consider app users engagement with certain app features known to improve clinical outcomes. Zhang et al. (Reference Zhang2019) examined patient engagement with health app features and found those who engaged more with features of learning, self-tracking, and goal setting had better clinical outcomes. The Wellness App offers goal setting and goal attainment, learning (helpful links and resources), and tracking (calendar) features. However, user engagement with learning and tracking features were not measured or considered for analysis given that we could not adequately track clinician support patients’ engagement with equivalent paper-based features and, therefore user engagement between groups could not be compared. There is a need to conduct rigorous efficacy trials with health apps to better understand the association between user engagement and health outcomes. Lastly, clinician intervention fidelity, an important implementation factor, was not measured. Despite these limitations, this study makes several novel contributions to the field of positive psychiatry and supports behavioral activation for brain health and wellness promotion as outlined above.
Conclusion
In conclusion, the current study found that in this applied positive psychiatry KT intervention, both the clinician-assisted and app-supported protocols led to improvements in patient outcomes in health behaviors and outcomes at the 4-week follow-up. This demonstrates promise for the implementation of a minimal brain health promotion behavioral activation intervention in frontline care. Of note, clinician-assisted patients had notably higher completion rates suggesting that clinician–patient relationships are of importance for intervention completion; lower completion rates among independent app users suggest a need to explore best practices for incorporating health apps in clinical practice in general, and in positive psychiatry interventions specifically. These results are timely and important for the development of effective health apps, adding to the currently small body of literature on health app use (Zhang et al., Reference Zhang2019).
Given the potential for positive psychiatry interventions to be delivered using health app technology, one avenue for future inquiry is whether and how health apps, used within clinician-assisted interventions (as a combination approach), might augment patient–clinician interactions and enhance patient health outcomes. Future research with more rigorous research designs, such as randomized controlled trials, are needed to substantiate the findings of this study. Another interesting area for future research would be to investigate the longer-term impact of these behavioral activation methodologies on behavior and attitude change. Additionally, further inquiry into the effectiveness of these clinical tools for health and well-being in at risk populations, such as caregivers, or older adults with mental illness, medical illness, frailty, mild cognitive impairment, or institutionalization, is needed.
An important contribution of this study is a method to promote effective goal setting and goal attainment in positive psychiatry domains to promote brain health and wellness using trained clinicians with access to behavior activation tools. Results suggest that clinician-assisted patients had “SMART-er” goals (i.e., goals that better adhered to the key “SMART-ness” items). While still in preliminary stages, the eight SMART-ness items outlined in this study add to positive psychiatry, by offering a practical starting point for future investigations into a robust goal-setting methodology for behavioral activation, to promote brain health and wellness.
Conflict of interest
None.
Description of authors’ role
K-L Cassidy, Co-Investigator, developed the Fountain of Health Clinician-supported and The Wellness App Health Behaviour Change Tools, designed the study intervention, supervised data collected, and wrote the paper.
M Vallis, Co-Investigator, developed the study question, was responsible for the statistical design of the study and for carrying out the statistical analysis, and wrote the paper.
D Conn, Co-Investigator, designed the study intervention, supervised data collection, and contributed to paper revisions.
A S Massie contributed to data collection and analysis and wrote the paper.
C Checkland, Co-Investigator, designed the study intervention, oversaw project management, and contributed to paper revisions.
D Parsons oversaw project management and contributed to paper revisions.
J Spence Mitchell contributed to data collection and analysis and contributed to paper revisions.
K Rabheru, Principle Investigator, project team lead, designed the study intervention, supervised data collection, and contributed to paper revisions.
