Prescribed assignments or homework between sessions is a crucial component of cognitive behaviour therapy (CBT) that is difficult to implement successfully (Helbig & Fehm, Reference Helbig and Fehm2004; Kazantzis, Arntz, Borkovec, Holmes, & Wade, Reference Kazantzis, Arntz, Borkovec, Holmes and Wade2010). Adhering to assignments is important since the completion of assignments has shown to be associated with positive treatment outcomes (Addis & Jacobson, Reference Addis and Jacobson2000; Mausbach, Moore, Roesch, Cardenas, & Patterson, Reference Mausbach, Moore, Roesch, Cardenas and Patterson2010). This association is reasonable given that the purpose of assignments is to foster behaviour change and to increase generalisation to other contexts than the therapy session (Kazantzis & Lampropoulos, Reference Kazantzis and Lampropoulos2002). Adherence to assignments increases if assignments follow logically from the presented treatment model and if they are congruent with the patient's treatment goals (Scheel, Hanson, & Razzhavaikina, Reference Scheel, Hanson and Razzhavaikina2004). Also, adherence is facilitated if the therapist and the patient collaborate on designing the assignments and if the therapist follows up on assignments and provides feedback (Cox, Tisdelle, & Culbert, Reference Cox, Tisdelle and Culbert1988; Tompkins, Reference Tompkins2002). Given the major role that assignments have in CBT, relatively few studies have examined additional strategies therapists can use in order to foster assignment adherence (Kazantzis, Whittington, & Dattilio, Reference Kazantzis, Whittington and Dattilio2010).
Many clinical studies do not report patients’ adherence to assignments in detail, but when reported, adherence to assignments are often moderate at best (Edelman & Chambless, Reference Edelman and Chambless1995; Simpson, Marcus, Zuckoff, Franklin, & Foa, Reference Simpson, Marcus, Zuckoff, Franklin and Foa2012). Patients’ reasons for less than optimal adherence to assignments in CBT include time restraints, executive difficulties, and competing priorities (Helbig & Fehm, Reference Helbig and Fehm2004). Some therapists probably use supportive e-mails or telephone appointments, but the clinical effects of such arrangements have only drawn moderate research attention, and they also consume valuable therapist time (Cucciare & Weingardt, Reference Cucciare and Weingardt2007). Standardised or automatic prompts, such as automatic text message reminders to patients’ mobile phones, have been shown to be effective in enhancing some general aspects of treatment adherence, including reducing dropout and increasing treatment engagement, though results are somewhat mixed (Clough & Casey, Reference Clough and Casey2011; Lindhiem, Bennett, Rosen, & Silk, Reference Lindhiem, Bennett, Rosen and Silk2015). Most studies have investigated the effects of prompts on adherence to medication or appointments, and only a small number of studies have focused on prompts to increase treatment engagement in psychotherapy. In a review, Fjeldsoe, Marshall, and Miller (Reference Fjeldsoe, Marshall and Miller2009) suggested that text-message features such as tailored content and interactivity may be important in text-message-based interventions, but in a more recent review, Orr and King (Reference Orr and King2015) did not detect any additional effects of tailoring text messages compared to standardised messages. They did, however, find that high-frequency text message interventions — that is, at least daily text messages — were more effective than low-frequency interventions for changing health behaviours.
Text messages used as a conjunction to face-to-face therapy have shown to improve treatment engagement and adherence in only a few studies. In a study by Aguilera and Muñoz (Reference Aguilera and Muñoz2011), text messages were used mainly to replace registration forms for assignments. The messages were thus designed to gather information as well as prompts in that study. Similarly, the text messages used in a study by Shapiro et al. (Reference Shapiro, Bauer, Andrews, Pisetsky, Bulik-Sullivan, Hamer and Bulik2010) fulfilled several functions as they were used to collect data on problematic behaviours and symptoms from the participants and to provide standardised feedback. Other studies have failed to find positive effects of text-message reminders for assignments (Furber, Jones, Healey, & Bidargaddi, Reference Furber, Jones, Healey and Bidargaddi2013; Shingleton et al., Reference Shingleton, Pratt, Gorman, Barlow, Palfai and Thompson-Brenner2016). For example, in the study by Shingleton et al. (Reference Shingleton, Pratt, Gorman, Barlow, Palfai and Thompson-Brenner2016), idiosyncratic text messages that were sent daily to participants in an eating disorder treatment did not significantly affect eating behaviour. However, it is worth noting that the text messages were used as reminders to treatment assignments, but the main outcome variable was treatment outcome, and adherence to assignments was not investigated specifically. The reasons for the mixed findings regarding the additive treatment effects of text messages are unknown, but to promote behaviour change it has been suggested that associating external prompts, such as text messages, with each patient's goals is crucial (Fogg, Reference Fogg2009). The very modest effects of mobile prompts seen in previous studies could be explained by an overreliance on external pressure when prompts should, for example, and similar to the principles of motivational interviewing, explicitly relate to the long-term goals of patients; but whether idiosyncratic text messages may have this effect has not been investigated (Baker & Hambridge, Reference Baker and Hambridge2002; Markland, Ryan, Tobin, & Rollnick, Reference Markland, Ryan, Tobin and Rollnick2005).
In conclusion, there is a need to investigate methods for increasing adherence to assignments in CBT (Kazantzis, Whittington, & Dattilio, Reference Kazantzis, Whittington and Dattilio2010). While personal between-sessions contact between therapist and patient may have some benefits, such arrangements are time consuming and unpractical in a clinical context. Using automatic prompts in patient contacts are much cheaper and less burdensome for therapists, but have mostly been evaluated for reminding of appointments and taking medications (Orr & King, Reference Orr and King2015). Prompting for more demanding patient behaviours, such as completing assignments, may call for more elaborate interventions (Jones et al., Reference Jones, Anton, Gonzalez, Honeycutt, Khavjou, Forehand and Parent2015).
The primary aim of this study was to investigate the effect of tailored text messages on adherence to assignments in a standardised CBT program. A secondary aim was to assess how the text messages were perceived by the participants and whether they reported any negative effects on treatment engagement. To reach these goals, we investigated the effects of text-message reminders in a relaxation program for people with elevated stress and/or worry. This intervention and population were chosen since regular relaxation is known to be an effective treatment for these common problems. Further, the exercises were of a type and format that is quite typical for a range of CBT assignments in that it is behavioural, fairly standardised, can be used frequently, and is possible to measure objectively.
Methods
Design
This study used a single-case experimental alternating-phase design with sequential replication (Blampied, Reference Blampied1999; Kratochwill et al., Reference Kratochwill, Hitchcock, Horner, Levin, Odom, Rindskopf and Shadish2012; Kratochwill & Levin, Reference Kratochwill and Levin2010; Manolov & Moeyaert, Reference Manolov and Moeyaert2016). The phase lengths and order were set for each individual by a software program using restricted randomisation. The study period of 35 days was randomly divided into a number of control (A) and intervention (B) phases between 3 and 8 days long for each participant. Control phases (A) contained no text messages while intervention phase (B) contained daily text messages. This design was chosen since the intervention was hypothesised to have limited carry-over effects (i.e., receiving a text message has a short-term effect on adherence).
There are no strict guidelines for conducting power analyses for this type of design, but statistical power can be assessed roughly based on the expected effect size, randomisation procedures, and the number of possible permutations (Ferron & Onghena, Reference Ferron and Onghena1996). In short, the modest effect sizes expected in this study could be offset by the experimental design, the randomisation procedure, the estimated large number of possible permutations and the hypothesised direct and non-lingering effect of the intervention.
A pre-study analysis suggested that six participants would be enough in this study to detect small to medium effects in adherence. To allow for dropout and missing data, it was decided to include seven participants in the study. During the study, all participants were blinded to the exact purpose of the study, and therapists were allowed to discuss the text messages in general, but not the specific effects of the text messages on assignment adherence unless brought up by the participants.
Treatment
The treatment consisted of a manualised five-week relaxation program for people with symptoms of stress and worry. The treatment was based on a previous treatment manual that has shown good effects on anxiety (Öst, Reference Öst1987) and was complemented with exercises in imagery techniques and mindfulnes (Hayes-Skelton, Roemer, Orsillo, & Borkovec, Reference Hayes-Skelton, Roemer, Orsillo and Borkovec2013). It had detailed instructions for each session and contained progressive muscle relaxation, release-only relaxation, deep breathing, positive imagery, mindfulness, and relapse prevention (Manzoni, Pagnini, Castelnuovo, & Molinari, Reference Manzoni, Pagnini, Castelnuovo and Molinari2008). Therapists and participants met for five weekly 45-minute sessions. Each session included psychoeducation, practising new exercises and the scheduling of daily exercises until the next session. In order to avoid possible ceiling effects and to increase internal validity, all participants were asked to do four exercises a day throughout the treatment. A clinical psychologist and a master-level psychology student provided all treatments, and all sessions were audio-recorded in order to assess treatment fidelity. In order to assess the clinical relevance of the treatment, self-report symptom scales were used before and after treatment.
The assignments consisted of daily exercises that were derived from the theme of each weekly session (e.g., progressive muscle relaxation for session 1) and the therapist and the client practised each exercise together once during the session before discussing the training schedule for the coming week. Though the exact content of the assignment changed each week, they were all designed to span 10–15 minutes up to four times a day so that the participants’ daily treatment workload would be constant during the study period.
Intervention
The intervention consisted of daily text messages that each participant received on their mobile phone. Before treatment start, the exact content of the text messages was decided collaboratively by each therapist and participant, but it had to relate to either a positive experience (e.g., a positive emotion) or a goal (e.g., the benefits of exercises), both of which may hypothetically increase the chances of positive reinforcement after an exercise (Michalak & Holtforth, Reference Michalak and Holtforth2006). Messages relating to external pressure (e.g., feeling obliged to do exercises), and therefore hypothesised to increase the risk of negative reinforcement, were not allowed. Examples of tailored text messages used in the intervention were ‘Do I feel stressed? Doing an exercise may make me feel better’ and ‘Doing an exercise may make it easier to focus on my studies’, corresponding to a positive experience and an idiosyncratic goal respectively. Each participant designed up to five different messages in order to appeal to different motivations and reduce habituation to the reminders. The timing and schedule for text messages were decided in collaboration between therapist and participant in order to minimise pressure and other negative perceptions of the messages. A computer program was used to administrate the text messages and to automatically send the prompts to participants' mobile phones.
Measures
The main outcome variable was adherence to the prescribed exercises in the treatment program. During the treatment, participants were asked to complete four exercises a day, which they registered on forms provided and collected by the therapists at each session. The number of registered exercises thus ranged from 0 to 4 per day during the treatment.
In order to ensure that the treatment was clinically relevant for participants, a number of self-report measures were used before and after the treatment to assess symptom improvements. Stress symptoms were measured with the Perceived Stress Scale-10 (PSS-10; Cohen, Kamarck, & Mermelstein, Reference Cohen, Kamarck and Mermelstein1983). The PSS-10 comprises 10 items that are scored on a scale from 0 to 4, providing a total score between 0 and 40. The PSS-10 has shown adequate psychometric properties in previous evaluations (Lee, Reference Lee2012).
Symptoms of worry were measured with the Generalised Anxiety Disorder-7 (GAD-7; Spitzer, Kroenke, Williams, & Löwe, Reference Spitzer, Kroenke, Williams and Löwe2006). The GAD-7 provides a total score of 0–21 and has shown adequate psychometric properties (Kroenke, Spitzer, Williams, & Löwe, Reference Kroenke, Spitzer, Williams and Löwe2010).
Treatment satisfaction was measured with the Client Satisfaction Questionnaire (CSQ-8; Attkisson & Zwick, Reference Attkisson and Zwick1982). The CSQ-8 consists of eight items and provides a score between 8 and 32, with a higher score representing higher satisfaction with the treatment. The CSQ-8 has shown to have adequate psychometric properties in previous studies (Attkisson & Greenfield, Reference Attkisson, Greenfield and Maruish1999).
Evaluation of the text-message intervention was measured with four questions in which the participants were asked to rate the degree to which they perceived the text messages as helpful, annoying, redundant and valuable on a scale from 0 (not at all) to 4 (absolutely). These questions provided a total satisfaction score between 0 and 16, and a score of 50% (≥8) or more of the total score was considered being satisfied with the intervention.
Treatment motivation was measured with the Situational Motivation Scale (SIMS), which was developed based on self-determination theory (SDT) to measure motivation in experimental tasks (Guay, Vallerand, & Blanchard, Reference Guay, Vallerand and Blanchard2000). The SIMS comprises four subscales — intrinsic motivation, identified regulation, external regulation, and amotivation — corresponding to the analog constructs described in SDT. The SIMS contains 16 items scored on a scale from 1 to 7, providing a score between 4 and 28 for each subscale. It has mainly been used in sport and health psychology and has shown adequate psychometric properties (Standage, Treasure, Duda, & Prusak, Reference Standage, Duda, Treasure and Prusak2003).
The short-form version of the Depression Anxiety and Stress Scale (DASS-21; Antony, Bieling, Cox, Enns, & Swinson, Reference Antony, Bieling, Cox, Enns and Swinson1998; Lovibond & Lovibond, Reference Lovibond and Lovibond1995) was used to screen for symptoms of depression, anxiety, and stress at study inclusion. The DASS-21 has previously shown adequate psychometric properties and is widely used in research (Henry & Crawford, Reference Henry and Crawford2005). The DASS-21 consists of 21 items with seven in each of the three subscales. Each item is scored on a scale from 0 to 3, providing a score between 0 and 21 for each subscale. Following the original recommendation, the cut-off scores used in this study for the depression, anxiety and stress subscales were 11, 7, and 7 respectively (Lovibond, Reference Lovibond1995).
Participants and Procedure
Participants were recruited at a university campus by advertisement on public billboards concerning health issues. The advertisements were aimed at people seeking help for symptoms of stress or worry. In order to identify people with elevated symptoms of stress or worry, all potential participants were asked to complete a semistructured clinical interview focusing on the most common mental health problems (i.e., mood and anxiety disorders), lifestyle, alcohol and drug use, and social or financial difficulties. They were also asked whether they currently had or previously had any treatment for stress, worry or anxiety. After being informed about the study, potential participants were asked to sign the informed consent before being included and asked to complete a set of self-report instruments. Inclusion criteria were symptoms of moderate stress or anxiety (DASS-21 stress or anxiety > 7) and using a mobile phone daily. Exclusion criteria were currently attending psychotherapy, conducting regular relaxation exercises, having severe symptoms of depression (DASS-21 Depression > 11; Lovibond, Reference Lovibond1995), <18 years old, showing severe levels of stress symptoms that warranted specialised care, or having major health-related or psychosocial problems (e.g., alcohol dependence). Any person who reported severe psychiatric symptoms was excluded and referred to clinical care.
Ten people reported interest in the study and were interviewed to assess study eligibility. Three of these were excluded after the interview. One person was excluded due to ongoing psychotherapy, one person was excluded due to being pregnant and having difficulties in prioritising participation, and one person was excluded for deviating substantially from the rest of the participants regarding the background variables of age and education. In single-case studies, the recommendation is to first use homogenous samples and then to continue with more heterogeneous samples in order to increase generalisability (Kazdin, Reference Kazdin2011). The remaining seven people were deemed eligible for inclusion in the study. Participants were between 20 and 32 years of age, all were university students, and all were Indigenous with high verbal proficiency. One participant had an ongoing medical treatment for anxiety, but since the medication had been stable for more than three months the person was not excluded from this study. Background variables and screening results for each participant can be seen in Table 1.
Background Variables and Screening Results for Each Participant
Note: F = Female, M = Male, Psych = Psychotherapy, Pharm = Pharmacological, DASS-21 = Depression Anxiety Stress Scale-21, Dep = depression subscale, Anxiety = anxiety subscale, Stress = stress subscale.
After inclusion, each participant was blindly randomised to an alternating phase schedule and then started the treatment. Data regarding adherence were collected each week during treatment, while all self-report instruments were collected before and after the treatment. At study end, all participants were asked to provide feedback on the study, including the text-message intervention. The study followed the Helsinki Declaration code of ethics and the study protocol was approved by the Regional Ethics Committee Board.
Statistical Analyses
Prior to further analyses, the trend, slope, and variability of each phase were analysed statistically to assess any differences between phases as well as any changes over time. Differences between the mean numbers of registered exercises in each phase were analysed with randomisation tests (Edgington & Onghena, Reference Edgington and Onghena2007). A randomisation test is a non-parametric statistical method that is suitable for single-case designs since there are no assumptions regarding data distributions or autocorrelation. Instead, randomisation tests are a type of permutation test that builds on the random assignment of the independent variable (i.e., intervention phases) to participants (Onghena & Edgington, Reference Onghena and Edgington1994). This analytic strategy was chosen over classical visual analysis, given that the design of the study and the primary outcome variables would make visual inspection very difficult.
For this study, the null hypothesis was that there would be no differences in registered exercises between the control and intervention phases. This can be investigated by ranking and comparing the mean values of the obtained phase differences measured in the study to the mean values of all possible data permutations in the design (e.g., see ter Kuile et al., Reference ter Kuile, Bulté, Weijenborg, Beekman, Melles and Onghena2009). The p value from this test is obtained by calculating the proportion of possible values that are equal or greater than the measured value. In contrast to statistical tests such as t tests, this p value can be calculated without making assumptions about a hypothesised distribution since the complete data distribution is used in the analysis. However, this is not feasible when the possible number of permutations is very large, as the calculations require enormous computational power. Therefore, when the number of permutations exceeds about 100,000, it is more practical to use Monte Carlo sampling methods to collect a random subsample for the calculations. To increase statistical power, this study used sequential replication with seven participants. This type of replication increases generalisability and enables the use of meta-analytical procedures such as combining p values (Solmi & Onghena, Reference Solmi and Onghena2014).
In this study, the procedures suggested by Heyvaert and Onghena (Reference Heyvaert and Onghena2014) were followed when analysing the data. First, the number of possible phase permutations was calculated for the chosen design and found to be >100,000. Second, one random phase permutation was picked for each participant using a digital random number generator. For example, the phase allocation for participant A's 35 days in the study was randomised to BBBBAAAAABBBBBAAAAABBBAAAABBBBBBAAA. Third, the intervention was provided and the data were collected. Fourth, the observed test statistic was calculated and since the mean value was expected to be higher in the B phases (text message intervention) compared to the A phases (no intervention), the test statistic M B – M A was used in all analyses. Fifth, the test statistic for all possible permutations was estimated (using a Monte Carlo random sampling method and a subsample of 1 000 permutations). Sixth, the p value for the collected data was calculated, that is, the proportion of the test statistics that were as extreme or more extreme as the collected test statistic was identified.
After this procedure had been conducted for each participant, p values for all participants were combined using the additive method. Non-overlap of all pairs (NAP) was used as a measure of effect size (Parker & Vannest, Reference Parker and Vannest2009). NAP is calculated by comparing each data point in phase A with each data point in phase B for each participant. The NAP is the proportion of all comparisons with no overlap between data points. The NAP is easy to calculate, especially for small data sets, and is comparable to R2, making comparisons between studies easier. Parker and Vannest suggest that NAP < .65 equals a weak effect size, NAP = .66–.92 equals a medium effect size and NAP = .93–1.0 equals a strong effect size. The R statistical software and the RcmdrPlugin.SCDA package were used for all analyzes (Bulté & Onghena, Reference Bulté and Onghena2008, Reference Bulté and Onghena2013).
Clinically significant change was estimated by calculating reliable improvement regarding symptoms of stress and worry for each participant (Jacobson & Truax, Reference Jacobson and Truax1991). Based on normative data, this corresponded to an improvement of 8 points on the PSS-10 (Nordin & Nordin, Reference Nordin and Nordin2013) and an improvement of 5 points on the GAD-7 (Löwe et al., Reference Löwe, Decker, Müller, Brähler, Schellberg, Herzog and Herzberg2008).
Results
Adherence to Assignments
All but one participant reported doing at least two exercises per day on average over the whole treatment period. The mean number of registered exercises for each phase and for each participant can be seen in Table 2.
The Mean Number of Registered Exercises for Each Phase and the Corresponding p Values and Effect Sizes for Each Participant
Note: Phase A = control phase, Phase B = intervention phase, NAP = Non-overlap of all pairs. acombined p value = .018.
The mean number of exercises was higher in the intervention than the control phase for all participants but one, and the combined p value for the phase difference in completed exercises for all seven participants was .018. The phase differences corresponded to medium effect sizes for participants C and D and to weak effect sizes for the remaining five participants. In order to investigate possible carry-over effects from the intervention phase to the control phase at each alternation, the analyses were also conducted with data from the first day after each phase alternation excluded. The mean number of exercises the first day after a change from phase B to A was not statistically higher than the mean of the control phases, indicating no carry-over effect of the intervention. In an ad-hoc analysis, these days were removed before analyses, which increased the intervention effect marginally but not the overall results and is therefore not presented here. The trend, slope, and variability in each phase was assessed for each participant and for all participants combined, but no significant differences between conditions nor any effects of time were found.
Treatment Effects
Regarding symptoms of stress and anxiety, four of seven participants reported clinically significant change regarding the PSS-10, and five of seven participants regarding the GAD-7. See Table 3 for symptom scale scores and treatment evaluation for each participant.
Self-Reported Symptoms of Stress and Anxiety at Pre- and Post-Treatment and Treatment Evaluation for Each Participant
Note: aClinically significant change. PSS-10 = Perceived Stress Scale-10, CSQ-8 = Client Satisfaction Questionnaire-8.
Intervention Evaluation and Treatment Motivation
The CSQ-8 ranged from 23 to 31 across participants, suggesting moderate to high satisfaction with the treatment (Attkisson & Greenfield, Reference Attkisson, Greenfield and Maruish1999). Four participants rated the text messages positively (a score ≥8) while three participants rated them as neutral or irrelevant (a score <8). In the open feedback section, positive feedback from participants included that text messages were helpful as reminders for exercises and for keeping the treatment on the agenda between sessions. Negative feedback included perceived pressure for not completing all assignments and that the text messages were not idiosyncratic enough. Despite some major changes in motivation in two participants, as measured with the SIMS at pre- and post-treatment, it was not possible to identify any patterns in these scores over time or any associations with treatment adherence or symptom reduction.
Discussion
The aim of this study was to investigate whether individually tailored daily text-message reminders would influence participants’ adherence to prescribed exercises in a brief treatment for stress and worry. The combined p value of .018 from all participants was significant, and individual analyses showed that two participants reported medium effect sizes while the remaining five participants reported weak effect sizes of the text message intervention. A moderate effect size in this study corresponded to completing about one half more exercise on the days with text messages compared to days without text messages. The results propose that between-session text messages can be marginally effective in improving adherence to assignments. It is noteworthy that in this study, the overall adherence to the assignments was high, which may have resulted in a ceiling effect. Participants reported completing a mean of about two exercises per day which, in the authors’ clinical experience, is somewhat above what is typically seen in this type of intervention. Reasons for this elevated baseline level are unknown but may be explained by participants being highly motivated or the very structured treatment protocol. When asking patients to register assignments, the registration in itself probably works as a reminder for the assignment. In clinical practice this may be beneficial, but it may pose a problem in research, since only the additive effect of text-message prompts can be investigated. The participants reported rather stable levels of motivation on the SIMS, with only two participants reporting large changes between pre- and post-treatment. This study was not primarily designed to investigate whether tailored text-message prompts may affect motivation, but it seems that the text messages at least did not increase perceived external control in an adverse way.
The results from this study and other recent studies are somewhat discouraging for those who argue that mobile technology may play a large part in future psychotherapy (Clough & Casey, Reference Clough and Casey2015; Jones et al., Reference Jones, Anton, Gonzalez, Honeycutt, Khavjou, Forehand and Parent2015). In the study by Shingleton et al. (Reference Shingleton, Pratt, Gorman, Barlow, Palfai and Thompson-Brenner2016), daily text messages failed to increase adherence to treatment prescriptions. In the current study, about half of the participants rated the text messages as helpful and half rated the text messages as irrelevant. However, all but one participant reported an increased number of completed assignments in the intervention phases, so the intervention may have had a subtle effect on most participants. The results indicate that the effects of mobile technology may vary somewhat across patients, though this needs to be confirmed in larger group studies.
Similar to the study by Shingleton et al. (Reference Shingleton, Pratt, Gorman, Barlow, Palfai and Thompson-Brenner2016), we used goal-oriented and tailored text messages as a complement to standard treatment, but while they found no effects on a group level, we found a marginal positive effect. The effects were weak to moderate and were probably found because of the high statistical power that was achieved by the randomised design in this study (Kratochwill & Levin, Reference Kratochwill and Levin2010). The current study used a prototypical CBT program, which comprised relaxation training, imagery techniques and mindfulness, and similar to the study by Shingleton et al., the homework assignments consisted of concrete behaviours. It is still unclear whether text messages may be more or less effective for other types of assignments such as thought records, exposure exercises or behavioural experiments that are common in CBT, but they may be more challenging or difficult to implement well.
All but one participant reported at least some improvement on the main clinical outcome measurement regarding stress symptoms, the PSS-10. Clinically significant improvement was reported by four participants regarding stress and by five participants regarding worry. Interestingly, the two participants who reported the largest effects of the text message intervention on relaxation training also reported clinically significant improvement in symptoms of stress and worry, as well as moderate and high satisfaction with the text messages. Unfortunately, this study was not designed to analyse mediation processes, so the associations between text messages, adherence to assignments, and treatment outcome is so far only hypothetical. Overall, the results on the clinical instruments show that the treatment was rather efficient for ameliorating the participants’ symptoms, and while relaxation may not be the most common treatment for symptoms of stress and worry, the results provide some support for the clinical relevance of this study (Manzoni et al., Reference Manzoni, Pagnini, Castelnuovo and Molinari2008).
There are many challenges for conducting successful single-case studies that we tried to address in the design of this study (Kazdin, Reference Kazdin2011; Kratochwill et al., Reference Kratochwill, Hitchcock, Horner, Levin, Odom, Rindskopf and Shadish2012). First, a well-defined and relevant outcome variable is of paramount importance and this study used completed assignments (e.g., relaxation exercises), since this variable is easy to measure, reliable and clinically relevant. Second, the intervention should have a substantial effect on the outcome variable as it makes interpretation of data easier and increases the likelihood of finding effects in small samples. In this study, the effect of text messages on adherence was unknown beforehand, but the study was designed to be able to detect even small effects. Third, interpreting and evaluating ambiguous results is difficult, especially with standard visual inspection. A randomisation test is one of the feasible alternative analytic methods for single-case data but there are few established guidelines (Solmi, Onghena, Salmaso, & Bulté, Reference Solmi, Onghena, Salmaso and Bulté2014). The study design and statistical analyses of this study are not widely used but followed the recommendations in the research literature. In a simulation study, Ferron and Ware (Reference Ferron and Ware1995) showed that the power for randomisation tests are generally low but increases with more elaborate experimental designs and phase allocations. Haardörfer and Gagné (Reference Haardörfer and Gagné2010) later concluded that despite this shortcoming, randomisation tests may be viable for studies that closely follows design guidelines, and this study was designed in line with these guidelines, though there are no previous studies of similar design to use as a reference. The best method for assessing effect sizes in single-case data is also debated and there are several alternatives to NAP used in this study (Parker & Vannest, Reference Parker and Vannest2009). Fourth, it is often advised that initial samples should be homogenous with regard to background variables and symptom levels, and that studies are continuously replicated in order to increase generalisability. While the participants in this study had similar age and education, there were few differences regarding both current and historical treatments for stress and worry. While the sample was rather homogenous regarding background variables, it may be more important to consider other variables, such as different people's view on using mobile technology in therapy and being prompted between sessions.
The results from this study suggest that text messages may be an effective method to marginally improve adherence to assignments in CBT. The reasons for less than optimal adherence among patients vary, and it may be important to identify each individual's obstacles and difficulties rather than evaluating standardised solutions. Prompts should, by definition, have no substantial long-term effect but rather work by initiating a behaviour that has not occurred spontaneously. Prompts and reminders may therefore be most efficient when the main reason for not conducting a behaviour is forgetfulness (Agyapong, Farren, & McLoughlin, Reference Agyapong, Farren and McLoughlin2011). Non-adherence to assignments is probably often a combination of forgetfulness and lack of adequate reinforcement. In this study, prompts were designed to remind each participant about the reasons for completing the assignments without creating external pressure. However, it is possible that the external pressure of text messages offsets any positive effects, at least for some people. For example, external pressure may make patients conduct the assignment in a mechanical fashion that fosters a quantitative view of homework compliance, which may not always be helpful (Kazantzis et al., Reference Kazantzis, Whittington, Zelencich, Kyrios, Norton and Hofmann2016). It may be fruitful to shift focus and investigate whether the quality of completed homework combined with a flexible use of prompts and prompts tailored for each individual can affect adherence to assignments.
While the results of this study show that text-message prompts can marginally improve adherence to assignments for patients in CBT, it is possible that increasing treatment motivation and engagement requires a more flexible use of mobile technology in psychotherapy.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/bec.2019.10.
Acknowledgments
This study was funded by the institution and none of the authors received an external grant for this project.
Conflicts of interest statement
The authors declare that there are no conflicts of interest.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
