Approximately 1 in 45 people have an acquired brain injury (ABI) in Australia, and its prevalence increases with age (AIHW, 2007). Memory is one of the most frequently affected cognitive capacities following brain injuries such as stroke (Garcia et al., Reference Garcia, Roussel, Bugnicourt, Lamy, Canaple, Peltier and Godefroy2013; Jokinen et al., Reference Jokinen, Melkas, Ylikoski, Pohjasvaara, Kaste, Erkinjuntti and Hietanen2015), traumatic brain injury (TBI) (Cristofori & Levin, Reference Cristofori and Levin2015; Ponsford et al., Reference Ponsford, Sloan and Snow2012), hypoxia (Garcia-Molina et al., Reference Garcia-Molina, Roig-Rovira, Ensenat-Cantallops, Sanchez-Carrion, Pico-Azanza, Bernabeu and Tormos2006; Turner et al., Reference Turner, Barker-Collo, Connell and Gant2015), encephalitis (Nicolle & Moses, Reference Nicolle and Moses2018) and some types of brain tumour (Scoccianti et al., Reference Scoccianti, Detti, Cipressi, Iannalfi, Franzese and Biti2012; Taphoorn & Klein, Reference Taphoorn and Klein2004). External memory aids can effectively compensate for various day-to-day memory difficulties (Rees et al., Reference Rees, Marshall, Hartridge, Mackie and Weiser2007; Sohlberg et al., Reference Sohlberg, Kennedy, Avery, Coelho, Turkstra, Ylvisaker and Yorkston2007). Among these, the use of electronic devices and assistive technology has attracted significant attention (Charters et al., Reference Charters, Gillett and Simpson2015; Jamieson et al., Reference Jamieson, Cullen, McGee-Lennon, Brewster and Evans2014). In particular, smartphones have been identified as promising tools, with various studies associating the use of mobile phones with improvements in self-initiated behaviours, task completion rates and functional independence (Charters et al., Reference Charters, Gillett and Simpson2015; de Joode et al., Reference de Joode, van Heugten, Verhey and van Boxtel2010; Ferguson et al., Reference Ferguson, Friedland and Woodberry2015; Jamieson et al., Reference Jamieson, Cullen, McGee-Lennon, Brewster and Evans2014; Svoboda et al., Reference Svoboda, Richards, Leach and Mertens2012; Wade & Troy, Reference Wade and Troy2001).
Despite the preliminary evidence supporting efficacy, adoption of technological devices as an external compensatory aid has been low (de Joode et al., Reference de Joode, van Heugten, Verhey and van Boxtel2010). From a clinician’s perspective, this low uptake has been attributed to insufficient training on how to teach compensatory strategies to individuals with brain injury and limited research regarding the most effective methods for doing so (Ehlhardt et al., Reference Ehlhardt, Sohlberg, Kennedy, Coelho, Ylvisaker, Turkstra and Yorkston2008; Fleming et al., Reference Fleming, Shum, Strong and Lightbody2005). From a client’s perspective, people with memory impairment can find it challenging to learn the use of a new device or app as it demands various cognitive processes, and it represents a memory task in itself (Wade & Troy, Reference Wade and Troy2001; Wilson, Reference Wilson2017). These difficulties have been previously identified by individuals with stroke and TBI as barriers to using smartphones in their daily life (Wong, Sinclair, et al., Reference Wong, Sinclair, Seabrook, McKay and Ponsford2017; Wong, Wang, et al., Reference Wong, Wang, Stolwyk and Ponsford2017).
Skill training methods within rehabilitation literature have usually focused on two approaches. The Trial-and-Error Learning (TEL) method, the traditional approach used in rehabilitation settings, promotes learner-initiated exploration of tasks and encourages people to attempt or guess responses. This approach assumes that people are able to remember and learn from their mistakes by analysing events and forming conclusions about their experiences in a self-initiated way. This, however, may not be the most effective method when memory and/or executive dysfunction prevents people from learning from their mistakes (Sohlberg & Turkstra, Reference Sohlberg and Turkstra2011). Systematic instruction (SI), on the other hand, is a method using a ‘bottom-up’ approach where the trainer makes use of different error-reducing techniques (i.e., errorless learning, chaining, modelling, cue fading) to prevent errors during the acquisition phase of learning (Riley & Venn, Reference Riley and Venn2015). There is some evidence that SI tends to achieve superior learning outcomes than TEL (Ehlhardt et al., Reference Ehlhardt, Sohlberg, Kennedy, Coelho, Ylvisaker, Turkstra and Yorkston2008; Kessels & de Haan, Reference Kessels and de Haan2003; Powell et al., Reference Powell, Glang, Ettel, Todis, Sohlberg and Albin2012); although it has been noted that its efficacy might vary across tasks and depend on severity of memory impairment. Verbally able individuals with severe explicit memory difficulties appear to benefit the most (Ehlhardt et al., Reference Ehlhardt, Sohlberg, Kennedy, Coelho, Ylvisaker, Turkstra and Yorkston2008; Evans et al., Reference Evans, Wilson, Schuri, Andrade, Baddeley, Bruna and Taussik2000). Results for people with less severe impairments are less conclusive with some studies reporting benefits (de Werd et al., Reference de Werd, Boelen, Rikkert and Kessels2013) and others finding no differences compared with TEL conditions (Bier et al., Reference Bier, Van Der Linden, Gagnon, Desrosiers, Adam, Louveaux and Saint-Mleux2008; Dunn & Clare, Reference Dunn and Clare2007).
It has been noted that skills learned with error reducing techniques may not generalise to untrained tasks (Wilson et al., Reference Wilson, Winegardner, van Heugten and Ownsworth2017). Error-based learning (EBL) is a ‘top-down’ approach that moves away from the principle of error prevention and utilises metacognitive skills training to promote error self-regulation and awareness of deficits, providing structured opportunities for people to learn from making errors (Ownsworth et al., Reference Ownsworth, Fleming, Tate, Beadle, Griffin, Kendall and Shum2017). Contrary to SI, EBL avoids the use of direct instruction. Instead, through the use of systematic feedback and prompts, people are trained to frequently stop, check and notice errors (SCaN steps), learn to correct these during task performance and to reflect on their meaning (i.e., awareness of deficits). The internal focus of this approach is thought to facilitate generalisation of skills beyond the trained task (Ownsworth et al., Reference Ownsworth, Fleming, Tate, Shum, Griffin, Schmidt and Chevignard2013). To date, only Ownsworth et al. (Reference Ownsworth, Fleming, Tate, Beadle, Griffin, Kendall and Shum2017) have compared the effectiveness of EBL with error-reducing techniques. After training participants with severe TBI in a meal preparation task and other goal-directed activities, EBL was found to be more effective than SI for transferring learned skills to a training-related task (i.e., near transfer) and increasing self-awareness. However, the skills did not generalise to unrelated tasks (i.e., far transfer) and differences were not maintained between the groups at a 6-month follow-up.
To date, research targeting the learning of electronic devices as memory compensatory aids has mainly compared error-reducing approaches with TEL, producing mixed results (Evans et al., Reference Evans, Wilson, Schuri, Andrade, Baddeley, Bruna and Taussik2000; Svoboda et al., Reference Svoboda, Richards, Leach and Mertens2012; Wilson et al., Reference Wilson, Baddeley, Evans and Shiel1994). To our knowledge, no previous studies have investigated the effectiveness of EBL for teaching the use of electronic aids in people with memory impairment after ABI. The potential of EBL to facilitate transfer to related tasks may be particularly relevant for smartphone apps, as these are often changed and updated, and similar errors can occur across apps.
Accordingly, the primary aim of this study is to compare the efficacy of three training methods (Systematic Instruction, Error-based Learning and Trial-and-Error) for training the use of a smartphone reminder app in ABI survivors presenting with memory complaints. Efficacy is defined as proficiency of performance in tasks with the trained app, and this will be measured at three time-points: immediately after the training and at 1- and 6-week follow-up, with the six-week follow-up being the primary timepoint. It is hypothesised that participants trained under SI and EBL will be more proficient than those trained with TEL at each time-point. It is further hypothesised that people trained with EBL will show greater skills generalisation, as evidenced by them completing more tasks accurately in untrained apps than participants in the other conditions. Other secondary aims of this study are to determine whether the type of training promotes changes in memory self-efficacy, confidence in using the smartphone and frequency of using the device in general and as a memory aid. Due to the exploratory nature of these aims, there are no specific hypotheses.
Methods
Study design
This is a three-armed Phase II randomised controlled trial (RCT) with blinded outcome assessments and between- and within-group comparisons. This study adheres to the PEDro practice guidelines for RCTs (Maher et al., Reference Maher, Sherrington, Herbert, Moseley and Elkins2003). Participants are equally allocated through covariate-adaptive randomisation (see randomisation section) to one of three training groups: SI, EBL or TEL.
Ethics
This study has been approved by the Monash University Human Research Ethics Committee (Project number: 11761) and has been registered in the Australian New Zealand Clinical Trials Registry (ACTRN12618001066279). Protocol amendments will be duly reported to these two bodies.
Participants and recruitment
Community-dwelling participants are recruited through multiple sources including (1) referrals from professionals in rehabilitation services; (2) advertisement in specific support groups and online platforms and (3) invitation to members of a stroke registry at Monash University who consented to be contacted regarding research opportunities. Candidates are eligible if they are older than 18 years, have had an ABI (e.g., stroke, TBI, benign or low-grade brain tumour in remission after treatment, encephalitis or hypoxic brain injury) at least 3 months before participating, have memory difficulties (self-reported or noted by clinician/family members), have a smartphone, are interested in learning a new app and live within travelling distance of metropolitan Melbourne. Exclusion criteria include inability to provide consent, significant aphasia, low proficiency in English, severe neglect, cognitive, motor or sensory difficulties limiting engagement in treatment and having another significant neurological or psychiatric condition impacting on functioning. Suitability for participation and technical requirements of the smartphone are confirmed at the start of the first session through a brief interview and cognitive screen.
App selection
All participants are trained to use the same app. App assessment and selection were guided by three main considerations: first, accessibility for research purposes (no cost and availability in iOS and Android operating systems); second, the recommendations proposed by Wild (Reference Wild2013) for evaluating apps when determining their appropriateness for people with ABI and third findings from previous surveys regarding the main benefits that patients with ABI perceive from smartphones as memory and organisational aids (Wong, Sinclair, et al., Reference Wong, Sinclair, Seabrook, McKay and Ponsford2017; Wong, Wang, et al., Reference Wong, Wang, Stolwyk and Ponsford2017). After shortlisting and ranking the apps and consulting expert clinicians, the selected app was Cozi Family Organiser (CoziInc., 2018). The main functionalities of this app include a calendar, reminders/alarms, the possibility to create lists and the option to share information with family members. Another two apps ranked highly in the shortlist, 24meLtd. (2018) and Wunderlist (WunderkinderGmbH, 2015) were chosen for the generalisation tasks. Out of these apps, only the 24me app requests access to information stored in the smartphone such as location, contacts and the in-built calendar. Since only access to the calendar is a requirement to use the app, participants are advised to deny other permissions. If a participant does not wish to allow calendar access, the 24me app is removed, and a research smartphone with the same operating system (resembling participant’s phone) is provided to facilitate completion of the tasks.
Intervention
The intervention procedures are based on the works of Svoboda et al. (Reference Svoboda, Richards, Polsinelli and Guger2010) and Ownsworth et al. (Reference Ownsworth, Fleming, Tate, Shum, Griffin, Schmidt and Chevignard2013). Protocols for each condition (TEL, SI and EBL) were developed and adapted to iOS and Android operating systems. All participants receive one training session during which four skills are trained with the Cozi app through four hypothetical scenarios: (1) creating and sharing a shopping list; (2) creating an event in the calendar; (3) modifying the shopping list and (4) checking and modifying the event during a phone call. The feasibility of the intervention and assessment protocol was piloted with healthy elderly people with changes leading to the final version that included (1) modifying the format of the materials provided to participants; (2) reducing the difficulty and amount of information provided in each scenario; (3) highlighting relevant information to be entered in the smartphone; (4) simplifying the instructions and prompts for each training method; (5) modifying assessment tasks to resemble the trained ones; (6) modifying threshold of maximum errors permitted per assessment task and (7) reorganising the order in which measures were collected.
A brief description of the training session and the key differences between the three training conditions can be seen in Table 1. Participants are trained according to their allocated condition; yet, they are not educated about the name of the conditions or the study hypotheses about whether or not their training condition is expected to be effective. Regardless of the allocation, all participants are assisted to download the app in their phones at the start of the session. Then, each scenario with its corresponding tasks is presented and executed one at a time. Other common elements in all three conditions include (1) positive verbal reinforcement after accurate performances; (2) written hypothetical scenarios that are read out aloud; (3) written step-by-step instructions on how to complete the tasks and (4) repetition of tasks. The number of repetitions varies depending on the training condition (please refer to the ‘Performance criterion’ row in Table 1), although a minimum of two and a maximum of four repetitions per scenario has been set across groups.
Overview of the Trial-and-error, Systematic Instruction and Error-based Learning Training Session
Treatment fidelity
The intervention is delivered by provisionally registered psychologists under the supervision of registered clinical neuropsychologists. Treatment fidelity strategies for optimising provider training and monitoring delivery of treatment are based on Bellg et al. (Reference Bellg, Borrelli, Resnick, Hecht, Sharp Minicucci, Ory and Czajkowski2004) and Borrelli (Reference Borrelli2011) and include (1) a three-phase standardised training process – introductory presentation with audio-visual material, independent video analysis and role-play – for providers of the intervention; (2) feedback on the first training session and periodic re-training; (3) standardised intervention protocols and materials; (4) minimisation of treatment contamination through a treatment implementation checklist and (5) video recording of all sessions to allow the examination of provider’s adherence to the protocol of a random sample (10%) of sessions by independent assessors. Some of the intervention providers are aware of the study hypotheses; however, it is expected that the above strategies to enhance treatment fidelity help ensure that the interventions are delivered according to standard protocol.
Measures
Baseline measures
The baseline measures are intended to confirm participants’ eligibility, determine overall baseline cognitive functioning and calculate covariate values for the randomisation process. During the first session, demographic and brain-injury-related information (i.e., date, location and mechanism of injury) is collected. Participants also complete the Montreal Cognitive Assessment (Nasreddine et al., Reference Nasreddine, Phillips, Bedirian, Charbonneau, Whitehead, Collin and Chertkow2005) to briefly screen their cognitive functioning.
Memory performance is assessed with the California Verbal Learning Test II Short form (Delis et al., Reference Delis, Kaplan, Kramer and Ober2008) and the Brief Visual Memory Test – Revised (Benedict et al., Reference Benedict, Schretlen, Groninger, Dobraski and Shpritz1996). To avoid ceiling effects in measures of verbal memory, people under the age of 50 are given the standard version of the California Verbal Learning Test II. Executive function is measured with the Hayling Sentence Completion Test (Burgess & Shallice, Reference Burgess and Shallice1997) and Trail Making Test (Reitan & Wolfson, Reference Reitan and Wolfson1985).
Smartphone ability is briefly determined with an activity specifically designed for this study. Participants are asked to complete seven basic tasks with the in-built functions of their phone (adding a new contact number, making a call, setting an alarm, taking a photo, sending a text message, adding an event in the calendar and downloading a new app). Each task is scored as completed or not, and a total score with the number of completed tasks is calculated.
Level of functioning in activities of daily living (ADL) is measured with the Nottingham extended ADL (NEADL) (Nouri & Lincoln, Reference Nouri and Lincoln1987), which is a 22-item, self-administered questionnaire about the level of independence in performing activities in four areas (mobility, kitchen, domestic and leisure). Although it was initially developed for stroke survivors, the NEADL has also been implemented in populations with other types of ABI (Das Nair & Lincoln, Reference Das Nair and Lincoln2012; Oort et al., Reference Oort, Taphoorn, Sikkes, Uitdehaag, Reijneveld and Dirven2019). Finally, mood is measured with the Hospital Anxiety and Depression Scale (HADS) (Zigmond & Snaith, Reference Zigmond and Snaith1983). Other measures that are administered at baseline are described in the secondary outcomes section.
Primary outcome measure
Proficiency of performance is determined by the total number of correct tasks independently completed by a participant in the Cozi app after the training intervention. This is measured at three time-points, immediately after the training and 1- and 6 weeks after, with the last follow-up considered the primary endpoint, and with a possible maximum score of 15 at each timepoint.
The 15 tasks are presented through hypothetical scenarios. They target the same skills trained during the intervention and involve creating lists (i.e., Create a new list called ‘Afternoon tea’), modifying lists (i.e., Adding flour and onion was a mistake of the recipe. Delete them from the list) and sharing lists of items (i.e., Share the list over email); as well as checking events (i.e., when are you available?), creating events (i.e., Add your dentist appointment to the calendar), modifying events in the calendar (i.e., Reschedule your original appointment) and using the Cozi app while on a phone call. Each time point has a different but equivalent version of the hypothetical scenarios and the 15 tasks, all of which are administered in the same order. Each single task is considered not completed if the participant commits seven errors while attempting the task or if the overall execution is incorrect.
The primary outcome measure is not administered at baseline because initially exposing participants to the Cozi app would resemble a TEL condition, which would negate the use of the SI approach, where error commission is prevented. To control for differences in baseline function, smartphone ability is balanced across conditions during the randomisation process.
Secondary measures
Generalisation of skills is determined by the total number of correctly completed tasks on two untrained apps (Wunderlist and 24me). This is measured at the 1- and 6-week follow-ups. A total of 10 tasks, presented through hypothetical scenarios, were also developed for the purposes of this study. Tasks are scored as successfully completed or not and there are two different but equivalent versions for each time-point (i.e., scenarios have fundamentally identical structures and the same actions are required, but the information to be entered is different).
The number of errors is calculated for each of the tasks completed with the Cozi app and the untrained apps at each time point. Actions that are considered errors include touching unnecessary areas of the screen, selecting the wrong buttons, failing to select buttons to confirm information (i.e., save, done), clicking on pop-up ads, omitting or deleting information and entering incorrect information. Steps taken to correct errors (i.e., selecting ‘cancel’ or ‘back’) or remove ads are not counted as errors, as well as minor spelling errors that do not change the meaning of information or accidentally clicking on a different area to the intended one due to dexterity issues. Since there is a limit of seven errors per task, there is a maximum of 105 errors for the tasks with the Cozi app and 70 errors for the untrained apps, at each time point.
Frequency of smartphone use as a memory aid is determined with one section of the Strategies of Smartphone Use questionnaire – SSUQ (Svoboda et al., Reference Svoboda, Richards, Polsinelli and Guger2010), which measures self-reported use of smartphone to access retrospective and prospective information, with 12 items scored on a 5-point scale ranging from 1 = ‘Never’ to 5 = ‘Always’.
Confidence in smartphone use is established with a modified separate section of the SSUQ. The single global confidence item (How would you rate your confidence in using the smartphone?) was extended into eight more items enquiring about confidence in using specific functionalities (i.e., ‘How would you rate your confidence in using your smartphone calendar?’). All nine items are scored on a 5-point scale ranging from 1 = ‘Not confident at all’ to 5 = ‘Very confident’. Scoring of both measures from the SSUQ entails the total sum of their corresponding items, and higher scores indicate higher levels of the construct.
General smartphone usage is defined as the amount of time, in hours, a participant uses the phone for any purpose within the last 7 to 10 days. For iOS users, data are taken directly from the participant’s phone auto-generated statistics, while for Android users the app Social Fever (Systweak, 2017) is downloaded to monitor overall usage. Usage information is accessed and recorded by the researcher in the presence of the participant.
Memory self-efficacy (MSE) is measured with the Memory Controllability Inventory –MCI (Lachman et al., Reference Lachman, Bandura, Weaver and Elliott1995), a 13-item questionnaire containing four scales (present ability, potential improvement, effort utility and inevitable decrement). It is rated on a 7-point scale ranging from 1 = ‘Strongly disagree’ to 7 = ‘Strongly agree’, where higher scores indicate higher MSE. The MCI has been reported to have a consistent factor structure and an internal consistency ranging from 0.58 to 0.77 for each subscale (Lachman et al., Reference Lachman, Bandura, Weaver and Elliott1995)
Finally, self-reported memory difficulties are assessed with the Everyday Memory Questionnaire-Revised (EMQ-R) (Royle & Lincoln, Reference Royle and Lincoln2008), which asks participants to rate statements regarding everyday memory functioning on a scale from 0 = ‘Once or less in the last month’ to 4 = ‘Once or more in a day’. It consists of 13 items, and higher scores indicate more everyday memory difficulties. The EMQ-R has a strong internal consistency of 0.92 (Royle & Lincoln, Reference Royle and Lincoln2008).
Randomisation
Given the relatively small sample size of this trial and the number of conditions, group allocation is conducted through covariate-adaptive randomisation to optimise the balance among groups with respect to three covariates: cognition (memory and executive functioning), age and current ability using the smartphone (Hu et al., Reference Hu, Hu, Ma and Rosenberger2014). The cognition variable is calculated as the sum of converted T-scores for memory and executive function tests. Age and the raw score for the Smartphone ability task are used without modifications. Given that all covariates are continuous, a variation of the method of biased coin randomisation proposed by Frane (Reference Frane1998) is implemented. Under this method, the first six participants are randomly allocated. After this, each new participant is temporarily assigned to each training group and a p value, using analysis of variance (ANOVA), is obtained for each of the three covariates for each condition. The lowest of the three p values indicates the least balanced covariate in each possible allocation. This method uses those minimum values to calculate the probabilities of assigning the new participant to each condition, thus providing an algorithm weighting the randomisation to prioritise the group with the greatest difference in covariates and keeping imbalance over groups as small as possible. If the difference between group size becomes more than three participants, the weighted randomisation is only applied to the two conditions with fewer participants after an independent samples t-test analysis.
Considering that one of the main recruitment sources is a stroke registry, it is possible that a considerable proportion of the sample comprises stroke survivors. This means that a separate statistical analysis comparing the three conditions could potentially be conducted with only the sample of stroke participants. However, if the size of this stroke sample is not sufficient, participants with different ABI aetiologies will be combined for statistical analyses. Anticipating these two possibilities, the above randomisation process is conducted separately for the stroke survivors and for people with other types of ABI. This allows for separate analyses to be conducted if appropriate; or alternatively, a similar number of participants with different ABI aetiology in each condition if the samples are combined. Given that the biased coin process aims to reduce differences in the covariates and that there is a mechanism to maintain a relatively equal number of participants in each condition, it is expected that this balance will be maintained if the samples are combined.
Group allocation is concealed as the randomisation is conducted after participants are recruited in the study. The randomisation is performed on a computer, by an independent researcher not involved with recruitment, intervention or assessment, who directly emails the allocation to the designated provider of training. Participants are masked to their allocation. Although they are aware of the existence of three interventions, the actual names of all training methods or the hypotheses associated with them are not disclosed until the end of the last follow-up session.
Procedure
A general overview of the procedure can be seen in Fig. 1. Participants are initially screened by the research team and provide verbal consent over the telephone, followed by written confirmation of consent at the start of the first session. All participants are seen over four sessions, which can be completed at their home or at university training clinics. During the first session, demographic and medical information is collected. Baseline and secondary outcome measures, except for tasks measuring generalisation of skills, are administered.
Procedure overview.
Participants are then trained on the use of the Cozi app during the second session under either SI, EBL or TEL conditions. Immediately after the training, a post-training test comprising similar scenarios to the trained ones is administered. Two follow-up sessions are conducted 1 and 6 weeks after the training to re-assess skills with the Cozi app, to administer other follow-up measures and to assess generalisation of skills. The post-training test and the follow-up sessions are conducted by an independent blinded assessor. Video recordings of 30% of randomly selected assessment sessions are rated by another assessor to determine inter-rater reliability in the scoring of correct tasks and number of errors.
Procedure overview
Table 2 outlines the time points at which each measure is administered or collected. At the end of the 6-week follow-up, participants are invited to complete a rating questionnaire and a semi-structured interview about their experience during and after the training session. The questionnaire was designed for the purposes of this study (Appendix A), and the interview schedule (Appendix B) was adapted from previous similar studies (das Nair & Lincoln, Reference das Nair and Lincoln2013; Withiel et al., Reference Withiel, Sharp, Wong, Ponsford, Warren and Stolwyk2018). These data will be analysed and reported separately in a qualitative study where the ratings are interpreted descriptively, and open-ended responses are analysed using thematic analysis with a critical realist approach (Braun & Clarke, Reference Braun and Clarke2006). Participants are also offered assistance to remove the apps from their phones if desired. They are debriefed and offered to be informed of the results upon completion of the study through email or mail. Participants are made aware, when they provide consent, that researchers aim to communicate the results of the study through journal publications and academic presentations.
Timing of Assessment of Outcome Measures
Sample size
Previous research has reported very large effects (d = 1.44) of SI over TEL (Powell et al., Reference Powell, Glang, Ettel, Todis, Sohlberg and Albin2012). Accordingly, a power analysis performed with G*Power (Faul et al., Reference Faul, Erdfelder, Lang and Buchner2007) revealed that the minimum sample size required for detecting significant between-group differences is 42 (80% power, α = 5%, effect size f = 0.5). Considering attrition rates of 15%–20% reported in other studies (Brkic et al., Reference Brkic, Shaw, van Wijck, Francis, Price, Forster and Rodgers2016; Mares et al., Reference Mares, Cross, Clark, Vaughan, Barton, Poland and Pomeroy2014; Powell et al., Reference Powell, Glang, Ettel, Todis, Sohlberg and Albin2012), this study aims to recruit 51 participants (17 each condition). Studies investigating instructional techniques in ABI populations have typically included 30 or fewer participants (Ehlhardt et al., Reference Ehlhardt, Sohlberg, Kennedy, Coelho, Ylvisaker, Turkstra and Yorkston2008). Hence, if the target sample size is not reached, the proposed analysis will focus on effect sizes and information will still be used to estimate parameters for a future trial.
Data management and statistical analysis
Data are kept confidential, and only researchers have access to it. Hardcopy information is stored in a locked filing cabinet, and electronic data are password protected and stored on secure servers. All participants are assigned an ID number, which can be re-identified with a codebook stored separately from the study data. Data are entered and analysed in SPSS (Version 26.0). Range and outlier checks for data values are conducted prior to analysis. Baseline characteristics are reported with frequency distributions and descriptive statistics (mean, range and standard deviation). Analyses are conducted using an intention-to-treat approach implementing the last observation carried forward method. Missing data are handled with pairwise deletion.
The efficacy of the training approaches, reflected in the proficiency of performance with the Cozi app, is compared with a 3 × 3 mixed ANOVA and post-hoc tests, where the training method is the between-subjects factor and the assessment timepoint (post-training, 1- and 6-weeks follow-up) the within-subjects factor. The same procedure is implemented for the analysis of the number of errors. Changes in the frequency of use of smartphone (in general and as memory aid), confidence using a smartphone and memory self-efficacy as well as the effect of the training approaches are examined with four independent 3×3 mixed ANOVAs. However, a multivariate ANOVA (MANOVA) might be considered if significant correlations are identified among some or all of these four dependent variables, provided other assumptions of the model are also met. Generalisation of skills is analysed with a 3×2 mixed ANOVA. Baseline, demographic or other variables with significant effects on the outcomes are treated as covariates in all relevant analyses.
It is acknowledged that multiple statistical analyses will be undertaken; however, given that this is a phase II RCT and the small sample size, an unadjusted α-level of 0.05 is set for all analyses to reduce the likelihood of Type II errors (although it is acknowledged that the risk of Type I errors remains). A data monitoring committee is not deemed necessary given the nature and low-risk level of the interventions (Lin & Lu, Reference Lin and Lu2014).
Discussion
The use of electronic devices, such as smartphones, has been recommended as a practice guideline for people with ABI (Charters et al., Reference Charters, Gillett and Simpson2015) and yet their adoption and implementation in rehabilitation settings is low (de Joode et al., Reference de Joode, van Heugten, Verhey and van Boxtel2010). This is potentially related to two factors. First, clinicians have no clear evidence-based guidelines regarding how to train smartphone use or the most suitable methods to do it. Second, some people who would benefit from smartphone use are not familiar with technology or/and have difficulties learning how to use them.
By comparing TEL, SI and EBL, this study will provide information regarding the most effective method to train people with ABI with memory difficulties in how to use apps that can be used as compensatory memory aids. In the process, some challenges are anticipated. The first potential issue is recruitment of participants. This type of intervention is very targeted; its particular focus on technology can affect recruitment as some people may prefer non-electronic compensatory memory aids or, alternatively, people interested in participating may not have a smartphone. Different means to advertise the project are implemented, as explained in the participants section. Particular attention is given to local support groups and referrals from community-based clinicians. The second potential issue is retention of participants for the follow-up sessions. This is addressed by scheduling appointments well in advance, obtaining at least two points of contact, offering home-based sessions, facilitating transportation and reminding participants one day before the session. Another challenge is practice effects if participants use the apps between sessions. To resemble real-life conditions, the use of the apps post-training is not encouraged or discouraged. Although between-session use of the apps might be an indicator of the efficacy of the intervention, it can also contribute to participants’ performance in the primary outcome measure above and beyond the single training session itself. This is taken into account during data collection as participants are asked about their usage of the apps in each session and time spent on them is recorded when possible. Lastly, it has been acknowledged that some intervention providers are aware of study hypotheses and this might be a limitation. This is addressed with the implementation of different strategies, such as standardised intervention protocols and materials, to monitor treatment fidelity and promote consistency in the delivery of interventions.
Relevant practical and theoretical implications are expected from this study. First, if our findings indicate efficacy of some or all of the training methods, this study could be used to inform a larger phase III RCT. Longer term results will not only contribute to the debate concerning the advantages or disadvantages of using SI and error-reducing techniques over conventional approaches within rehabilitation such as TEL but will also determine the potential of using EBL as an emerging training method for people with memory impairment. Future research could investigate predictors of outcome for each type of intervention, including severity of impairment in memory and executive functions. The effect of these training methods on long-lasting adoption of smartphones, levels of functional independence, self-esteem, mood and community participation could also be explored. Our intervention and findings will support trainers/therapists involved with memory rehabilitation programmes and have the potential to be extended to other training targets and, more broadly, across other populations with memory complaints.
Trial status
Recruitment commenced in November 2018 and it is still ongoing.
Authors’ contributions
All authors contributed intellectually to the study design and were involved in the drafting of the manuscript and/or critically revising the intellectual content. DR had the lead role preparing the manuscript, designing the study and interventions as well as the recruitment process, delivery of interventions and ethics application. RS and DW supervised the project and ethics application and contributed with the study design, selection of outcome measures, development of intervention procedures, statistical analysis and recruitment process. DW also led the conceptualisation and rationale of the project. TO provided input on the intervention design and delivery process. All authors read and approved the final manuscript.
Acknowledgements
The authors would also like to thank Lisa Johnston who has kindly collaborated with the randomisation process.
Financial support
This project is being funded by the Turner Institute for Brain and Mental Health at Monash University.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Abbreviations
- TEL:
-
Trial and error
- SI:
-
Systematic instruction
- EBL:
-
Error-based learning
Competing interests
Diana Ramirez Hernandez has no conflicts of interest to disclose. Rene Stolwyk has no conflicts of interest to disclose. Dana Wong has no conflicts of interest to disclose. Tamara Ownsworth has no conflicts of interest to disclose.
Appendix A: Rating Questionnaire about Participants’ Training Experience
Appendix B: Interview Schedule of the Semi-Structured Interview
1. What were the best elements of the training session with your smartphone?
2. What were the worst elements of the training session with your smartphone?
3. How did you feel during the training?
4. What effects have you noticed on your life as a result of the training with your smartphone?
5. What effects has the smartphone training had on everyday activities where you need to plan or remember information?
6. Is there anything else you would like to say about your smartphone training experience?
Standard prompts
That is interesting, can you tell me more about that?
Can you expand a little bit more on that?
What do you mean by that?
Anything more related to the training itself?
Could you please give me some examples of that?
Is there anything else you like to add for this question?
If specific details are not given, feel free to modify the prompts above to obtain more information
