Introduction
Technology-assisted treatment, incorporating websites, smartphone applications, and self-paced self-management strategies, offers the promise of increasing access to and reach of potentially effective interventions in psychiatry. Different types of technology platforms may be appropriate for different purposes and users, and multiple interventions may be appropriate for individuals with complex, chronic conditions such as schizophrenia (Nasi, Cucciniello, & Guerrazzi, Reference Nasi, Cucciniello and Guerrazzi2015; O'Shea, McGavigan, Clark, Chew, & Ganesan, Reference O'Shea, McGavigan, Clark, Chew and Ganesan2017). Here, we report outcomes for the Center for Medicaid Services-funded Improving Care and Reducing Cost (ICRC) study (Clinical Trials #NCT02364544) that used multiple technologies to prevent psychiatric hospitalization for patients with schizophrenia who had recently been discharged following an inpatient admission.
Schizophrenia is an important disease target for chronic condition management. Psychiatric hospitalization is a major driver of cost (Ascher-Svanum et al., Reference Ascher-Svanum, Zhu, Faries, Salkever, Slade, Peng and Conley2010), relapses and persistent positive symptoms impede recovery and therapeutic alliance (Cavelti, Homan, & Vauth, Reference Cavelti, Homan and Vauth2016; Cavelti et al., Reference Cavelti, Winkelbeiner, Federspiel, Walther, Stegmayer, Giezendanner and Homan2018; Winkelbeiner et al., Reference Winkelbeiner, Cavelti, Federspiel, Kunzelmann, Dierks, Strik and Homan2018), and decrease patient quality of life (Almond, Knapp, Francois, Toumi, & Brugha, Reference Almond, Knapp, Francois, Toumi and Brugha2004). For individuals with schizophrenia, poor treatment adherence, discontinuation, and service rejection are common (Higashi et al., Reference Higashi, Medic, Littlewood, Diez, Granström and De Hert2013). These characteristics drive the need for a novel multicomponent approach to patient engagement and relapse prevention (Baumel et al., Reference Baumel, Correll, Hauser, Brunette, Rotondi, Ben-Zeev and Kane2016; Brunette et al., Reference Brunette, Rotondi, Ben-Zeev, Gottlieb, Mueser, Robinson and Kane2016): The 6-month ‘health technology program’ (HTP) included in-person, individualized relapse prevention planning that involved treatments delivered via smartphones and computers, as well as a web-based prescriber decision support program (Fig. 1a).
Health technology program reduces rehospitalization. (a) Study design. After discharge from the psychiatric hospital across 10 US sites, patients were non-randomly assigned for 6 months to either the standard relapse prevention group (SRG) or the health technology program (HTP). A mental health technology coach (MHTC) guided the patient through the program, providing coaching, monitoring, and close coordination with a psychiatric prescriber. Note that HTP included (1) contacts with the MHTC to develop and maintain a relapse prevention plan; (2) an interactive smartphone illness self-management system providing coping strategy training and brief interventions to improve medication adherence, mood regulation, sleep, social functioning, and coping with auditory hallucinations; (3) a daily support website for patients and relatives focusing on psychoeducation; (4) a web-based, self-administered, self-paced cognitive behavioral therapy for psychosis (CBTp) sessions for coping with voices and coping with paranoia; and (5) evidence-based pharmacological treatment. (b) Significant reduction in days spent in the hospital after discharge. Compared to the SRG, the 6-month HTP significantly reduced the mean stay in psychiatric hospitals after discharge. Mean cumulative difference in days during 6 months with 95% confidence intervals is shown.
A ‘mental health technology coach’ (MHTC) worked in a team to provide the HTP. Based on the stress-vulnerability coping model (Mueser, Corrigan, & Hilton, Reference Mueser, Corrigan and Hilton2002) and supported by smartphone- and computer-based treatments, the MHTC developed a personalized relapse prevention plan with the patient (Brunette et al., Reference Brunette, Rotondi, Ben-Zeev, Gottlieb, Mueser, Robinson and Kane2016). Since previous studies suggest that relapse prevention planning is an effective approach for patients with schizophrenia (Mueser et al., Reference Mueser, Corrigan and Hilton2002), we hypothesized that HTP compared to standard relapse prevention would decrease rehospitalization days during the 6-month intervention period after a recent hospitalization compared to usual care. Note that data from the CATIE study (Lieberman et al., Reference Lieberman, Stroup, McEvoy, Swartz, Rosenheck, Perkins and Hsiao2005) have documented that the 6 months following an admission for a psychotic episode is a period of high risk for rehospitalization and thus was the chosen period we examined in the study.
Materials and methods
Study participants and design
Inclusion criteria were age between 18 and 60 years; diagnosis of schizophrenia, schizoaffective disorder, or psychotic disorder not otherwise specified; discharge from psychiatric hospitalization within the past 60 days; ability to participate in research assessments in English; and ability to provide informed consent. Patients with a serious general medical condition were excluded. Note that having WIFI and a personal computer were not inclusion criteria. To enhance participation equity, we eliminated the potential for participation being restricted by participant financial constraints by providing a smartphone and a personal computer to participants while they were in the study.
Medication data were collected at study entry, 3 months, and 6 months. Supplementary Table S1 presents the percentage of participants receiving antipsychotic treatment divided into critical categories at each medication data collection time period. At study entry, the percentage of control and intervention participants receiving any antipsychotic, clozapine, and long-acting injectable antipsychotics (LAIs) was very similar; there was no evidence for differences between any proportion. The use of LAIs and clozapine was more common than in many outpatient settings and this probably reflects that participants had had at least two inpatient admissions for psychotic episodes prior to study entry.
The study was conducted at 10 sites in eight US states. Following a quasi-experimental design, up to 10 patients at each site were enrolled and assessed to form a control group starting on 26 February 2013. After completion of the control group enrollment, patients were enrolled in the experimental condition until 17 April 2015. Note that each site recruited control subjects and subsequently intervention subjects. Thus the control and intervention participants came from the same facilities. All patients provided written informed consent. The Feinstein Institutional Review Board (IRB) approved and monitored the overall study; if required by a given site, a local IRB also approved the study.
Intervention
Details of the intervention have been previously published (Baumel et al., Reference Baumel, Correll, Hauser, Brunette, Rotondi, Ben-Zeev and Kane2016; Brunette et al., Reference Brunette, Rotondi, Ben-Zeev, Gottlieb, Mueser, Robinson and Kane2016). Briefly, the 6-month HTP provides in-person, individualized relapse prevention planning that directs use of technology-based treatments delivered on smartphones and computers and provides a web-based prescriber decision support program.
Prescriber Decision Support System
Pharmacotherapy guidelines were implemented in HTP by using a computerized clinical decision support system with a measurement-based care approach, modeled on a system developed for the RAISE-ETP study (Recovery After an Initial Schizophrenia Episode Early Treatment Program), which was funded by the National Institute of Mental Health. All HTP prescribers and clients accessed the program from a secure website. Prior to each prescriber visit, the client entered information into the system by answering a series of simple questions about symptoms, side effects, and treatment preferences. The prescriber then reviewed these data, which guide the in-person evaluation session. The system includes recommendations about evidence-based medication strategies that inform joint client–prescriber decision making, as well as suggestions for other HTP interventions (e.g. ‘Coping With Voices’ for clients with auditory hallucinations).
Relapse prevention plan
The case manager and client collaboratively develop a personal plan by exploring reasons for preventing relapses; reviewing the last relapse; identifying early warning signs, triggers, and preventive strategies; making a plan to monitor and respond to early warning signs; identifying specific coping strategies; and seeking input from natural supports (e.g. family). A brief manual and worksheets supported the process of creating, learning, and sharing the plan with other service providers and supportive persons, as well as using the plan to guide deployment of technology-based treatments.
Smartphone intervention for schizophrenia
FOCUS is a smartphone system that has five modules which are designed to enhance coping with symptoms, mood, or sleep problems; improve social situations; and encourage medication use. Participants can select FOCUS content on the basis of persistent symptoms or sources of stress that increase risk of relapse. FOCUS uses system-generated audio prompts and on demand resources (visual aids, photos, and cartoons) and written text to teach evidence-based coping strategies through interactive assessments, exercises, and suggestions that clients can access as they encounter challenges in their daily lives.
The Daily Support Website
The Daily Support Website (DSW) uses both web- and phone-based resources to support illness management and recovery for persons with schizophrenia and their supporters. The program provides education about schizophrenia, teaches problem-solving skills, and facilitates social support for clients and their relatives and other supporters to reduce family stress and improve coping in order to prevent relapses. The DSW includes web-based group forums, a library of educational resources, an ‘ask the expert’ service, a ‘frequently asked questions’ library, and a news feed. The forums (one each for clients and supporters and one for both) are facilitated by therapists who promote and emphasize discussions on solving problems, alleviating stress, and increasing social support.
Computerized Cognitive Behavioral Therapy for Psychosis (CBTp)
Two web-based CBTp skills programs, ‘Coping With Voices’ and ‘Coping With Paranoia’, incorporate traditional elements of CBTp, including goal setting, psychoeducation, normalization of symptoms, coping strategy enhancement, cognitive restructuring, and home practice. The programs help clients develop strategies to cope with voices and paranoia to prevent symptoms from escalating to a relapse. The programs are interactive and self-paced. They include animated tutorials, exercises, and games; personal symptom and goals tracking; and independent skills practice worksheets. Participants progress through modules, cumulatively building skills.
Whether sessions were prompted by an investigator or were initiated by the participants depended upon the intervention. Specifically, the DSW that provided support and education to patients and families and the CBT program for voices and paranoia were participant initiated. The FOCUS smartphone app gave prompts to participants but also had participant-initiated features. The computer support system for medication decision making was incorporated into the routine patient–prescriber visit.
The intervention was flexible regarding which components participants employed. Participants were provided an introduction to all the intervention components and they could choose their level of participation. Participants usually had monthly prescriber visits but could have more frequent visits based upon clinical need. All participants had an individualized relapse prevention plan. This usually took four sessions to develop and one later to review at the end of study participation. Of the remaining interventions, the smartphone app is the simplest to use and is usually the technology tool that is first used by participants. Detailed use patterns for the smartphone app have been previously published (Achtyes et al., Reference Achtyes, Ben-Zeev, Luo, Mayle, Burke, Rotondi and Kane2019; Ben-Zeev et al., Reference Ben-Zeev, Scherer, Gottlieb, Rotondi, Brunette, Achtyes and Kane2016).
Primary outcome
The primary outcome measure was days spent in a psychiatric hospital during 6 months after discharge, as assessed by a monthly patient interview.
Statistical analysis
We hypothesized that participants in the HTP compared with the control group would have fewer days of hospitalization. We used a linear mixed model with inverse probability of treatment weighting (IPTW) to estimate the treatment effect. We then used these stabilized weights to weight each participant's outcome, the number of days spent in a psychiatric hospital, which was the primary outcome measure, and used treatment [standard relapse prevention group (SRG) v. HTP] and number of visits as predictors. We also included predictors for all the covariates that were used to calculate the propensity scores, including age, gender, race, diagnosis, socioeconomic status, study site, quality of life at baseline, cannabis and alcohol abuse, age at illness onset, age at first hospitalization, number of prior hospitalizations, length of last hospitalization, and days in medical treatment. A custom contrast comparing the difference of cumulative days spent in the hospital over the 6 months period between groups was then calculated. The estimate of this treatment effect is reported as raw estimate (difference in days) with 95% confidence intervals. We also report balance diagnostics of the IPTW (Austin & Stuart, Reference Austin and Stuart2015) which compare the baseline covariates before and after applying inverse propensity weights. We considered successful removal of confounding conditional on the IPTW by differences between covariates that were smaller than Cohen's d = 0.1 for continuous covariates and smaller than Cramer's V = 0.1 for categorical covariates (Austin & Stuart, Reference Austin and Stuart2015). Finally, we also tested for an effect of HTP on quality of life at month 6, using an IPTW weighted linear regression with the same covariates as in our primary analysis (see above).
Results
From the participants assigned to the SRG group, a total of 89 participants had complete data for propensity matching and were thus used in the final study. From the participants assigned to the HTP program, 349 participants had complete data for propensity matching and were used in the final study. Weighting the baseline covariates with inverse propensity scores resulted in a more balanced study sample (Austin & Stuart, Reference Austin and Stuart2015), with all covariates showing effect size differences smaller than Cohen's d = 0.1 and Cramer's V = 0.1, respectively (online Supplementary Fig. S1). In addition, both groups showed high rates of study completion, with a higher rate in the HTP group that did not reach statistical significance [χ2 (1) = 2.9, p = 0.088; Table 1].
Sample characteristics of the ICRC study
ICRC, improving care and reducing cost; SRG, standard relapse prevention group; HTP, heath technology program.
Over 6 months and compared to the SRG group, the HTP reduced the days of hospitalization by 5 days (mean days: b = −4.58, 95% CI −9.03 to −0.13, p = 0.044; Fig. 1b). Mean estimated days of hospitalization during 6 months for the SRG group were 16.21 and 11.64 for the HTP group.
To confirm that this effect was robust to the skewed distribution we repeated our analysis with a log-transformation of the days in hospital (to attenuate the skew) and confirmed that there were fewer hospitalization days during the HTP program compared to control treatment (b = 0.38, s.e. = 0.16, t = 2.42, p < 0.05).
In terms of ratios of participants who were rehospitalized with respect to the two groups, there were 38/89 (43%) patients rehospitalized from the control group and 83/349 (24%) patients from the HTP group. Note that this difference in rehospitalized patients was significant (χ2 = 11.76, p < 0.001). Thus, the proportion of rehospitalized patients was significantly higher in the control group than in the HTP group, confirming our main finding.
We also examined the influence of age by looking at the effect of age in our model. Interestingly, while we did not find that this effect was significant, the sign of the effect was negative, suggesting that if anything older patients spent fewer days at the hospital (b = −0.03, s.e. = 0.02, t = −1.75, p > 0.05).
Finally, using quality of life at month 6 as outcome, we found no significant effect of HTP [β = 0.02, t (345) = 0.43, p = 0.668].
Discussion
Recently hospitalized patients with schizophrenia who received an integrated technology-informed relapse prevention program (HTP) experienced fewer days in the hospital compared to those who received usual care in the 6 months following their discharge. Given the high patient burden and costs of even a single day spent in a psychiatric hospital, estimated at $1358 per day based on inflation-adjusted results from a recent study (Rosenheck et al., Reference Rosenheck, Leslie, Sint, Lin, Robinson, Schooler and Kane2016), our findings imply total savings in psychiatric inpatient expenditures of $6216 during the first 6 months after discharge on average, suggesting potentially important health-economic implications (Ben-Zeev, Razzano, Pashka, & Levin, Reference Ben-Zeev, Razzano, Pashka and Levin2021), even after additional intervention costs are taken into consideration. Technology-enhanced relapse prevention that is tailored to the individual patient may improve recovery and the ability of individuals to remain in the community, and reduce the costs for the management of schizophrenia significantly, offsetting technology access costs. In addition, the high acceptance and satisfaction reported previously (Baumel et al., Reference Baumel, Correll, Hauser, Brunette, Rotondi, Ben-Zeev and Kane2016) suggest that these patients are willing and able to engage in a novel and technology-enhanced approach to relapse prevention. This approach offers potential to overcome the sometimes low satisfaction with services reported by patients.
The question of whether technology-assisted treatment is more readily and effectively used by different population subgroups is one interesting and important question. For example, the cultural image of individuals who use technology extensively is that of young men. One could also hypothesize that educated compared to less educated participants would be more likely to use technology-assisted treatment. The FOCUS smartphone app recorded detailed metrics of use to test these associations. An analysis of high utilizers of FOCUS previously published (Achtyes et al., Reference Achtyes, Ben-Zeev, Luo, Mayle, Burke, Rotondi and Kane2019) identified that high utilizers were indeed more likely to be more educated but contrary to cultural expectations were more likely to be women than men and older compared with younger individuals. However, the main result of the current study was not influenced by age.
Schizophrenia is usually chronic and characterized by relapse. Crucially, relapse may reduce future treatment response and contribute to treatment resistance (Emsley, Nuamah, Hough, & Gopal, Reference Emsley, Nuamah, Hough and Gopal2012; Takeuchi et al., Reference Takeuchi, Siu, Remington, Fervaha, Zipursky, Foussias and Agid2018; Wiersma, Nienhuis, Slooff, & Giel, Reference Wiersma, Nienhuis, Slooff and Giel1998), highlighting the need for interventions after hospitalization that improve outcomes and reduce health care cost. Although technology-based intervention studies for patients with schizophrenia have shown promise (Alvarez-Jimenez et al., Reference Alvarez-Jimenez, Alcazar-Corcoles, Gonzlez-Blanch, Bendall, McGorry and Gleeson2014; Rotondi et al., Reference Rotondi, Anderson, Haas, Eack, Spring, Ganguli and Rosenstock2010), the current study suggests that engagement with technological interventions requires the support of trained personnel (Mohr, Cuijpers, & Lehman, Reference Mohr, Cuijpers and Lehman2011).
Four features of HTP were likely to drive the beneficial impact of the intervention (Baumel et al., Reference Baumel, Correll, Hauser, Brunette, Rotondi, Ben-Zeev and Kane2016). First, HTP included relapse prevention planning, cognitive-behavioral symptom management strategies, and family psychoeducation strategies, all of which are evidence-based and address common symptoms and distress. Second, in-person assistance through the MHTC may have enhanced the use of the digital tools. Third, patient's ratings suggest that the design of the tools was helpful (e.g. easy to use for those with cognitive deficits common in schizophrenia). Finally, the HTP approach to relapse prevention was flexible, with most contacts between the MHTC and patient occurring via technologies such as mobile phones, text, and e-mail.
While the coordinated specialty care (CSC) model (Jones et al., Reference Jones, Gius, Daley, George, Rosenblatt and Shern2020) has transformed the standard of in-person care for first-episode schizophrenia (emphasizing a team-based, multi-element approach to address the complex and varied needs of this population), significant barriers remain, a crucial element of which is cost (Ben-Zeev, Buck, Kopelovich, & Meller, Reference Ben-Zeev, Buck, Kopelovich and Meller2019). As shown in this report, technology does not have to exclude human interactions, but instead can be additive to human-delivered, evidence-based treatment, and most importantly can reduce costs. Consistent with CSC, HTP technologies are emphasizing a multi-component approach: they connect the client with others to create a team of community case managers, employment specialists, therapists, and people with lived experience that can help them succeed. Thus, HTP is a prime example of emerging technologies that may serve as the key scaffold to implementing effective practices in a scalable, personalized, and sustainable manner.
Importantly, evidence suggests that individuals with schizophrenia do use electronic devices, and they use it in a manner that is comparable to those without schizophrenia (Ben-Zeev et al., Reference Ben-Zeev, Buck, Kopelovich and Meller2019). In addition, there is strong evidence suggesting that younger generations – so-called digital natives (Prensky, Reference Prensky2001) who grew up with the Internet and with smartphones – will be capable and willing to use familiar technologies for mental health (Abdel-Baki, Lal, D-Charron, Stip, & Kara, Reference Abdel-Baki, Lal, D-Charron, Stip and Kara2015; Lal, Dell'Elce, & Malla, Reference Lal, Dell'Elce and Malla2015).
The current study has some limitations that merit comment. One cannot rule out that treatment globally may have become more sophisticated over a 2-year period. However, there are several factors that suggest that important changes in treatment during a 2-year period are unlikely. First, ICRC was done at 10 sites widely dispersed across the United States. Thus, treatment changes would have had to occur not locally but across the United States to be important factors in the study outcomes. Nation-wide changes are possible but occur very slowly with the diversified health care system in the United States. Second, no major new treatment paradigms for patients with multi-episode psychosis were released during the study period. We and all clinicians wish that major advances comparable to earlier advances such as the development of clozapine had occurred during the study period but unfortunately impactful treatment advances rarely occur.
Another limitation is that we did not use patients' own phones or computers. However, as troubleshooting of technical issues becomes more complex with multiple types of computers and smartphones, this was greatly improved by providing the same phones and computers to all participants. This greatly facilitated fixing technical issues as one has to solve issues with only one phone and one computer instead of the myriad of different models if using participant provided equipment. Further, regarding future clinical implementation of a HTP, developers can proactively tailor solutions to particular issues (e.g. the cellphone service provider upgrading cell phone operating systems remotely) knowing that solutions are needed for only a particular smartphone and computer.
Clearly, the non-randomized study design is an additional limitation of this study. However, data on high satisfaction and low drop out which we have presented in a previous publication (Baumel et al., Reference Baumel, Correll, Hauser, Brunette, Rotondi, Ben-Zeev and Kane2016) showed that 93% of enrolled patients completed the 6-month program, and 89% completed the program review; and that 72% were very satisfied and 24% were somewhat satisfied with the intervention. Thus, the high satisfaction and usability reported previously (Baumel et al., Reference Baumel, Correll, Hauser, Brunette, Rotondi, Ben-Zeev and Kane2016) as well as low dropout suggests that HTP is a feasible and potentially effective way to increase engagement with treatment and reduce hospitalization days in schizophrenia. Consequently, technology-based treatments (Ben-Zeev et al., Reference Ben-Zeev, Brenner, Begale, Duffecy, Mohr and Mueser2014) that involve the support of trained personnel (Ben-Zeev, Drake, & Marsch, Reference Ben-Zeev, Drake and Marsch2015; Mohr et al., Reference Mohr, Cuijpers and Lehman2011) may be an efficient alternative to conventional relapse prevention approaches that are hampered by the limited availability of highly trained staff. More rigorously controlled research is needed to evaluate the effects of the HTP program, and other technology-informed programs for reducing relapses and rehospitalizations for persons with schizophrenia at risk for rehospitalization. Obviously, there are relevant concerns about the use of technology in mental health care, including privacy breaches and the fear of replacing human-delivered care; such concerns must be addressed. However, if addressed there is also great potential for technology to improve mental health care, whether employed independently or with the assistance of a care team. In addition, although a cost-effectiveness analysis would have been valuable that was not built into this project and we did not track the exact number of visits; we were focused on feasibility and clinical impact. Finally, while there were multiple tech tools available to the participants and to the treatment providers, we did not collect a single measure that represents overall use of all the tools.
Much of current medical practice involves treatment of chronic conditions that impair patients in multiple domains and thereby require complex interventions. Schizophrenia, a disease often associated with motivational, cognitive, and executive dysfunctions, is an example of a condition with such treatment complexities. Given the success of the HTP model for participants with schizophrenia, treatment models with multiple technologies that allow for individualization of services, which are coordinated and facilitated by a health coach, may be feasible in the management of other chronic diseases requiring complex management strategies.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291722000794
Data
The data and code of this study are shared online to ensure reproducibility, doi: 10.17605/OSF.IO/YHVCE.
Acknowledgements
Supported by grant 1C1CMS331052-01-00 from the Centers for Medicare and Medicaid Services, Department of Health and Human Services. The contents of this report are solely the responsibility of the authors and do not necessarily represent the official views of the US Department of Health and Human Services or any of its agencies.
Conflict of interest
Drs Hauser and Rotondi have been consultants to Otsuka. Dr Brunette holds a research contract with Alkermes. Dr Ben-Zeev has been a consultant for eQuility and has had an intervention content licensing agreement with Pear Therapeutics. Dr Achtyes has served on advisory boards for Alkermes, Janssen, Karuna, Lundbeck/Otsuka, Roche, Sunovion, and Teva and reports previous stock holdings in AstraZeneca, Johnson & Johnson, Moderna, and Pfizer. Dr Achtyes has received research support from Alkermes, Astellas, Biogen, Boehringer-Ingelheim, InnateVR, Janssen, National Network of Depression Centers, Neurocrine Biosciences, Novartis, Otsuka, Pear Therapeutics, and Takeda. Dr Schooler has served on advisory boards or as a consultant for Abbott, Alkermes, Amgen, Eli Lilly, Forum (formerly EnVivo), Janssen Psychiatry, Roche, and Sunovion. She has received grant or research support from Genentech, Neurocrine, and Otsuka. Dr Robinson has been a consultant to Asubio, Costello, Innovative Science Solutions, and Shire, and he has received grants from Bristol-Myers Squibb, Janssen, and Otsuka. Marcy is a shareholder in Pfizer. Dr Kane has been a consultant for Alkermes, Eli Lilly, EnVivo Pharmaceuticals (Forum), Forest, Genentech, Intracellular Therapeutics, Janssen Pharmaceutica, Johnson and Johnson, Lundbeck, Neurocrine, Newron, Otsuka, Pierre Fabre, Proteus, Reviva, Roche, Saladex, Sunovion, and Teva. He has received lecture honoraria from Genentech, Janssen, Lundbeck, and Otsuka. He is a shareholder in LB Pharmaceuticals, and Vanguard Research Group. All other authors have no conflict of interest.
