Introduction
Employment of people with mental illness has become a global concern in the 21st century. Over the last two decades, evidence has accumulated on the effects of supported employment, particularly the individual placement and support (IPS) model, mainly across Western countries (Bond et al. Reference Bond, Drake and Becker2012; Kinoshita et al. Reference Kinoshita, Furukawa, Kinoshita, Honyashiki, Omori, Marshall, Bond G, Huxley, Amano and Kingdon2013). However, on average, around half of individuals in the past studies gained their employment even in the IPS model (Bond et al. Reference Bond, Drake and Becker2012). Therefore, clinicians and researchers have sought more effective services for people with mental illness, particularly those with low cognitive functioning (McGurk & Mueser, Reference McGurk and Mueser2004; Tsang et al. Reference Tsang, Leung, Chung, Bell and Cheung2010). A recent meta-analysis and a recent randomized controlled trial (RCT) emphasized that cognitive remediation and supported employment (CR + SE) has greatly boosted these individuals’ vocational outcomes (Chan et al. Reference Chan, Hirai and Tsoi2015; McGurk et al. Reference McGurk, Mueser, Xie, Welsh, Kaiser, Drake, Becker, Bailey, Fraser, Wolfe and McHugo2015). No existing studies, however, have examined the cost-effectiveness of this approach, even though recent trial-based studies on economic evaluation have separately reported positive results for supported employment and heterogeneous findings for cognitive remediation (Patel et al. Reference Patel, Knapp, Romeo, Reeder, Matthiasson, Everitt and Wykes2010; Heslin et al. Reference Heslin, Howard, Leese, McCrone, Rice, Jarrett, Spokes, Huxley and Thornicroft2011; Knapp et al. Reference Knapp, Patel, Curran, Latimer, Catty, Becker, Drake, Fioritti, Kilian, Lauber, Rossler, Tomov, Van Busschbach, Comas-Herrera, White, Wiersma and Burns2013; Hoffmann et al. Reference Hoffmann, Jackel, Glauser, Mueser and Kupper2014; Reeder et al. Reference Reeder, Harris, Pickles, Patel, Cella and Wykes2014). The lack of such data on CR + SE may be a substantial obstacle for practitioners and policy makers considering implementation of these services. The present study aimed to assess the cost-effectiveness of a CR + SE programme on vocational outcomes and cognitive function for people with mental illness and low cognitive function, compared with traditional vocational services (TVS).
Method
Design
We conducted an individual-level unblinded RCT with a parallel-group design and 1:1 allocation ratio at six sites in Japan. The trial included two arms (CR + SE group v. treatment-as-usual group). Outcome assessment for this economic evaluation took place at baseline and at follow-up 12 months after baseline. All sites were located in urban areas in four Japanese prefectures: Tokyo, Chiba, Miyagi and Kyoto. This study was approved by the Research Ethics Committee at the National Centre of Neurology and Psychiatry (no. A2011-024).
Participants
The eligibility criteria for study participants were as follows: (1) hospital out-patients at one of the six sites; (2) age 20–45 years; (3) diagnosis of schizophrenia, major depression or bipolar disorder (International Classification of Diseases-10); (4) cognitive dysfunction, with scores for the verbal fluency or coding subscales of the Brief Assessment of Cognition in Schizophrenia Japanese language version (BACS-J) at least –0.5 standard deviations below the average score of a healthy population (Keefe et al. Reference Keefe, Goldberg, Harvey, Gold, Poe and Coughenour2004; Kaneda et al. Reference Kaneda, Sumiyoshi, Keefe, Ishimoto, Numata and Ohmori2007); and (5) unemployment. The potential participants were given a full description of the study and the ethical issues involved, and written informed consent was obtained.
Sample size
The sample size calculation was conducted based on a past study which reported a 33% difference in the employment rate between the supported employment group and a control group (Oshima et al. Reference Oshima, Sono, Bond, Nishio and Ito2014). Detecting this difference with 90% power at a 5% significance level (two-sided) would require 43 people in each group. With a 20% attrition rate, the final estimation was approximately 52 in each group (104 in total).
Randomization
After participant recruitment and baseline BACS-J were complete, the research manager (S.S.), who was independent from the study interventions and analyses, randomized participants with stratification by sex (male v. female) and the score bands of the BACS-J (<−0.5 s.d. v. ⩾−0.5 s.d.). The research manager informed case managers in each site about the result of randomization. Then, the case managers informed individual participants about their allocation (CR + SE group or TVS group). Due to the nature of this study, blinding of allocation was impossible for participants and service providers.
Interventions
The participants in the CR + SE group received cognitive remediation in psychiatric day-care or community employment service agencies during the first 3 or 4 months. Psychiatric day-care is one of the services commonly used by people with mental illness in Japan; it generally provides social skills, recreation and a daytime place to stay, but does not focus on employment services, acute care to prevent hospital admission, or care for only people with severe mental illness. Community employment service agencies, in general, provide a long-term group-based work readiness training for people with mental illness before they undergo a job search, and are not integrated with medical services. Whether the intervention sites provided CR + SE through psychiatric day-care or community employment service agencies depended on which kind of services the hospitals, where the participants were recruited, already had in each site.
A cognitive remediation programme based on the ‘Thinking Skills for Work Program’ (McGurk et al. Reference McGurk, Mueser and Pascaris2005) and our previous study (Sato et al. Reference Sato, Iwata, Furukawa, Matsuda, Hatsuse and Ikebuchi2013) was constructed using CogPack software (Marker, Reference Marker2014). The programme involved two CogPack sessions per week over 12 weeks (24 sessions in total); tasks included those focusing on attention, concentration, psychomotor speed, learning, memory and executive functions. In addition, after one of CogPack session each week, participants engaged in hour-long verbal group sessions (total: 12 sessions) during which they discussed the importance of cognitive skills, performing activities of daily living and the development of compensatory strategies for managing persistent cognitive problems (Sato et al. Reference Sato, Iwata, Furukawa, Matsuda, Hatsuse and Ikebuchi2013). The verbal sessions aimed at linking the gap between individuals’ learning outcomes in the computer-based cognitive remediation programme and social/employment skills in their lives (Mogami, Reference Mogami2007).
Supported employment services were provided to participants after completion of cognitive remediation. In this study, these services partly incorporated the service principals of the IPS model (Becker & Drake, Reference Becker and Drake2003); for example, place-then-train approach, employment specialists in treatment teams, services based on individuals’ preferences and time-unlimited support. The services particularly included strength-based assessment, outreach services for living and work support, rapid job-search, family support and support for job continuation. Strictly, our supported employment programme was not IPS model due to unique employment laws in Japan, and we did not use the IPS fidelity scale. As an example of the way in which our programme differed, IPS staff directly contacted employers regarding job development and job placement. However, in Japan, one must be legally permitted to negotiate with companies concerning job development and placement for job-seekers (e.g. staff in unemployment offices), and thus staff in this study often introduced companies to their users through unemployment offices. During the study period, the staff in the CR + SE group received the 1-day training for five times to learn skills for cognitive remediation and supported employment.
In the TVS group, participants received the usual employment services currently used in Japan; these are based on the brokerage care management model. In this group, the care managers in the hospitals met the participants at least once a month and engaged with them and with community facilities that provided the traditional employment services. Most such facilities in Japan employ the train-then-place/step-wise model that attempts to improve work-readiness in people with mental illness and supplies training and simple tasks before actual job hunting. To avoid contamination between groups, the participants in the TVS group could not use psychiatric day-care/community employment service agencies providing CR + SE in each area during the study period.
Economic evaluation and perspectives
The economic evaluation was conducted from the perspective of a health care and social service system. These include the costs of hospital care, social services and (local) government services.
Vocational and clinical outcomes
The employment status and work tenures (days) of each participant were tracked daily or monthly by the care manager during the 12-month follow-up, using monitoring worksheets and the Client Service Receipt Inventory-Japanese version (CSRI-J) (Yamaguchi et al. Reference Yamaguchi, Shimodaira, Yoshida, Sato, Takahara, Maeda, Ichikawa, Izumida and Ito2012). The secondary outcome of this study, the cognitive function of participants, was evaluated using the BACS-J at baseline and at 12-month follow-up. A composite score was used to present the BACS-J results, since this is a standard international method and provides more readily understandable results than the BACS raw score. Each category score was standardized based on the mean score in a healthy population; this indicates the difference between the sample score and a mean score of ‘1’, which represents a healthy population (Keefe et al. Reference Keefe, Goldberg, Harvey, Gold, Poe and Coughenour2004; Kaneda et al. Reference Kaneda, Sumiyoshi, Keefe, Ishimoto, Numata and Ohmori2007). The composite score was then calculated by summing each standardized category score. We also assessed the baseline and 12-month scores of the Positive and Negative Syndrome Scale (PANSS), Hamilton Rating Scale for Depression (HAM-D) and Global Assessment of Functioning (GAF), in order to show the clinical condition of participants (Hamilton, Reference Hamilton1960; Kay et al. Reference Kay, Fiszbein and Opler1987; American Psychiatric Association, 1994). Clinical outcomes were assessed by participants attending doctors or trained psychologists at each site, none of whom was masked to allocation concealment.
Measures for use of services and cost
The medical receipts for cost of medical services were collected monthly by the research collaborators, who were masked to the group allocation. In Japan, the price of medical services and medications are statutorily recorded in patients’ medical receipts. The data of other service utilizations were collected by individuals’ care managers using the CSRI-J during monthly interviews. The CSRI-J collects data on not only vocational outcomes but also a wide variety of service utilizations, including social services (e.g. community employment services such as supported employment and TVS, community living support and accommodation), other services from public sectors and the use of medical services in hospitals/clinics from which we could not obtain participants’ medical receipts. In addition, the staff who provided supported employment services recorded the times and hours of activities that they could not bill for under relevant laws. For example, the current Japanese law on social services only covers the fees of follow-up services within 6 months after individuals are employed.
Costs of medical services and medications for psychiatric symptoms and side effects were computed using participants’ medical receipts as required by the National Health Insurance (Ministry of Health, Labour and Welfare, 2012b ). Social services including employment services were valued using unit costs derived from the Services and Supports for Persons with Disabilities Act (SSPDA), financial year 2012 (Ministry of Health, Labour and Welfare, Reference Medalia and Richardson2012a ). These costs include the elements for salaries, overheads and training. As the cognitive remediation was performed by psychiatric day-care or community employment service agencies, the utilization costs of these services (agencies) were applied to the cost of cognitive remediation.
With regards to information on unit costs that staff or organizations could not obtain under current laws, we applied the appropriate unit cost information specified in the National Health Insurance and the SSPDA (Ministry of Health, Labour and Welfare, 2012a , b ). In addition, for costs of services from public sectors (e.g. city office, local government and public unemployment office), we estimated the cost per hour based on the average salaries earned by public officers of average age in Japan (National Personnel Authority, 2013).
Statistical analysis
To assess group or site differences in sample characteristics and outcomes at baseline and 12-month follow-up, we used the χ2 test, Fisher's exact test and t test. In addition, we conducted logistic regression adjusting for site and baseline GAF score. To estimate the overall effects of CR + SE on the BACS-J, GAF and PANSS, we conducted a linear mixed model with restricted maximum likelihood (REML) adjusting for time, site, baseline score of each measure and baseline GAF score. Due to the small sample size, we conducted a repeated-measures analysis of variance for HAM-D. We also compared the mean change over time in each group's BACS-J composite score for cost-effective analysis.
All costs are presented in US dollars ($1 = 119 Japanese Yen on 1 September 2015). The means and standard deviations of the costs in each group were calculated at 12-month follow-up. The mean total cost, medical cost, intervention cost and cost of other services were compared between groups using mean differences and 95% confidence intervals (CIs), which were estimated from non-parametric bootstrap regressions (1000 repetitions). We conducted two regression models, one without adjustment and another with adjustment for several covariates, namely the cost of the first month excluding intervention costs, sites, and the following other variables which might be associated with vocational outcomes according to a past meta-analysis: age, sex, diagnosis, years of education, working experience and baseline BACS-J and GAF scores (Tsang et al. Reference Tsang, Leung, Chung, Bell and Cheung2010). The latter model constitutes a sensitivity analysis of economic evaluations to show the cost differences between the groups and the probabilities of CR + SE being cost-effective after controlling for covariates (McCrone et al. Reference McCrone, Craig, Power and Garety2010).
We also calculated incremental cost-effectiveness ratios (ICERs), which demonstrate the relationships between costs and employment rates (an additional 1% of people who worked at least 1 day), mean employment tenure (an additional day of work tenure) and mean improvement in BACS-J composite score (a change of 0.1 illustrated improvement). Each ICER was calculated as the difference in mean costs between the two groups divided by the difference in group outcomes.
Cost-effectiveness acceptability curves using the net benefit approach were employed to test sampling uncertainty in the probability that CR + SE is a cost-effective option when levels of willingness to pay vary for an additional unit improvement (Löthgren & Zethraeus, Reference Löthgren and Zethraeus2000). This approach shows the theoretical value (λ) that society is willing to pay for an additional unit of vocational and cognitive function benefit gained by CR + SE. The net benefit to society can be expressed as:
$${\rm Net \; benefit} = (\lambda \times E)-C,$$
where E is the effectiveness of outcomes and C is the service cost. To select λ, net benefits were estimated for all participants in the analysis assuming different values ranging from $0 to $300 in $10 increments for a 1% increase in the number of people working and an additional day of work tenure, and from $0 to $2000 in $100 increments for a 0.1 mean improvement of the BACS-J composite score. We calculated the net benefit values for each individual with a range of λ for each outcome. Then, coefficients for differences in net benefit values between the groups were derived from bootstrapped linear regressions (1000 repetitions) stratified by group. The proportion of the resulting coefficients greater than zero, presenting the probability that CR + SE was the optimal choice, was calculated for each λ. Cost-effectiveness acceptability curves were produced by plotting these probabilities. The regression models with covariate adjustments were performed as the secondary analysis. We also conducted a regression analysis with covariate adjustment using only the schizophrenia subsample. The third author (N.H.), who was masked to the group allocation, conducted all the analyses using Stata version 13 (USA).
Ethical standards
All procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Results
Participants
A total of 111 individuals with severe mental illness were randomly allocated to either the CR + SE group (n = 57) or the TVS group (n = 54), and 47 participants in each group were followed up for 12 months (attrition rates: 18% and 13%, respectively). In term of available data for those who dropped out (n = 3), their characteristics and most of the baseline scores of clinical measures were not significantly different from the entire participant group. Their baseline score of just the PANSS negative domain (21.67, s.d. = 9.29, t = −2.17, p = 0.032) was significantly higher than that of the whole sample. Cost data for 45 participants in the CR + SE group and 47 in the TVS group were available for economic analysis (Fig. 1). All the CR + SE group participants who were included in the analyses at least attended 80% of the sessions for the cognitive remediation programme, although four took 5 months to complete the programme. In addition, since five participants did not attend the BACS-J at 12-month follow-up, 87 participants were included in the economic analysis when using the BACS-J.
CONSORT (Consolidated Standards of Reporting Trials) flow diagram. BACS-J, Brief Assessment of Cognition in Schizophrenia Japanese language version.
Table 1 shows participants’ sociodemographics and clinical conditions. In both groups, approximately 60% were males, and over 80% (n = 80) were diagnosed with schizophrenia. The mean age was approximately 35 years old. The GAF score (54.43, s.d. = 11.23) in the TVS group was significantly higher than that in the CR + SE group (49.73, s.d. = 8.12, t = −2.288, p = 0.025). There were no significant differences in other sample characteristics or clinical conditions between the randomization arms at baseline (Table 2).
Demographics and clinical conditions at baseline for participants
CR + SE, Cognitive remediation and supported employment; TVS, traditional vocational services; s.d., standard deviation; CPZ eq, chlorpromazine equivalents.
a Mean of CP per day was only calculated from participants with a diagnosis of schizophrenia (n = 80).
Vocational and clinical outcomes at baseline and 12-month follow-up
CR + SE, Cognitive remediation and supported employment, TVS, traditional vocational services, s.d., standard deviation; OR, odds ratio; CI, confidence interval; BACS-J, Brief Assessment of Cognition in Schizophrenia Japanese language version; GAF, Global Assessment of Functioning; PANSS, Positive and Negative Syndrome Scale; HAM-D, Hamilton Rating Scale for Depression; REML, restricted maximum likelihood.
a Competitive employment: number of people who worked at least 1 day.
b OR was estimated form logistic regression adjusting for the site and the baseline score of GAF.
c Mann–Whitney test also found a significant difference (z = 4.069, p < 0.001).
d B for the BACS-J and PANSS were estimated from linear mixed model with REML adjusting for time, the sites, the baseline GAF score and the baseline score of each outcome measure.
e B for GAF were estimated from linear mixed model with REML adjusting for time, the sites and the baseline GAF score.
f F value and p value were estimated from one-way repeated-measures analysis of variance and effect size was estimated for group× time interaction.
Outcomes
As shown in Table 2, the employment rate during the study period [number of people working (n) = 28, 62.2%] was significantly higher in the CR + SE group than in the TVS group (n = 9, 19.1%, χ2 = 17.739, p < 0.001) (odds ratio = 11.06, 95% CI = 3.53 to 34.62, p < 0.001 after controlling for site and baseline GAF score). The mean work tenure in the CR + SE group (78.62 days, s.d. = 88.44) was also longer than in the TVS group (24.87 days, s.d. = 66.28). Over 12-month follow-up, there were significant improvements in the BACS-J composite score (B = 0.21, 95% CI 0.11–0.31, p < 0.001) in the CR + SE group relative to the TVS group. The CR + SE group exhibited a mean improvement in the BACS-J composite score of 0.32 relative to the TVS group (CR + SE: 0.55, TVS: 0.23, 95% CIs 0.12–0.52, t = 3.191, p = 0.002). In terms of site differences in outcomes, only the employment rates in the CR + SE group significantly differed between sites (χ2 = 11.423, p = 0.044). Detailed information on vocational outcomes and cognitive outcomes at each site is shown in online Supplementary Table S1. In the schizophrenia subsample, data were derived for employment rate [CR + SE: number of people working (n) = 24, 63.2%; TVS: n = 8, 19.0%], mean work tenure (CR + SE: 82.8 days, s.d. = 90.5; TVS: 23.3 days, s.d. = 64.7), and mean improvement in the BACS-J (CR + SE: 0.56; TVS: 0.28). Detailed information on the schizophrenia subsample is shown in online Supplementary Table S2.
Cost
The two groups showed different patterns in service use over the follow-up period (online Supplementary Table S3). Basically only the CR + SE group utilized psychiatric day-care as an intervention site, with a mean of 17.0 times (s.d. = 24.9) compared with 0.4 (s.d. = 1.3) in the TVS group. The mean number of in-patient night stays were 2.3 (s.d. = 10.2) in the CR + SE group and 16.5 (s.d. = 50.9) in the TVS group, respectively. The re-hospitalization rates were 6.7% (n = 3) in the CR + SE group and 17.0% (n = 8) in the TVS group.
The total costs and the costs for each category are summarized in Table 3. The total mean cost in the CR + SE group ($9823, s.d. = $6372) was lower than that in the TVS group ($11 063, s.d. = $11 263). This difference was not statistically significant, even after adjusting for covariates (B = −$1640, 95% CI −$5559 to $2279, p = 0.412). A clear difference was seen in the mean cost of medical services between the groups (B = −$3979, 95% CI = −$7816 to −$143, p = 0.042, adjusted for covariates), due to the costs of in-patient services (CR + SE: $560, TVS: $3578). The mean intervention cost ($4202) in the CR + SE group was slightly higher than that in the TVS group ($2915), although the employment service cost was very similar between the groups (CR + SE: $2850, TVS: $2915). Detailed information about monthly costs is shown in online Supplementary Tables S4 and S5. The total mean costs in the schizophrenia subsample were $9722 (s.d. = $6548) in the CR + SE group and $11 750 (s.d. = $11 677) in the TVS group.
Mean costs ($) of the CR + SE and the TVS over 12 months
CR + SE, Cognitive remediation and supported employment; TVS, traditional vocational services; s.d., standard deviation; MD, mean difference; CI, confidence interval; GAF, Global Assessment of Functioning; BACS-J, Brief Assessment of Cognition in Schizophrenia Japanese language version.
a $1 = ¥119 (1 September 2015).
b MD = B estimated from non-parametric bootstrap regression.
c Covariates: sites, age, sex, diagnosis, education years, working experience, baseline BACS-J score, baseline GAF score, and the first month cost excluding intervention for total cost.
d Covariates: sites, age, sex, diagnosis, education years, working experience, baseline BACS-J score, baseline GAF score, and the first month costs of medical services, interventions or other social services.
Cost-effectiveness
The CR + SE group was dominant for employment rate (ICER = −$29), employment tenure (ICER = −$23), and the 0.1 improvement in the BACS-J (ICER = −$387), compared with the TVS group. Figs 2–4 show cost-effectiveness acceptability curves for each outcome. With an x axis value of zero, the cost-effectiveness acceptability curves for gaining an additional 1% of people working and an additional work day show around a 70% likelihood of the CR + SE being more cost-effective than the TVS group. In the schizophrenia subsample, there is over an 80% likelihood of the CR + SE being more cost-effective for gaining an additional work day, with no additional payment. In each curve (no adjustment, adjustment for covariates, and schizophrenia subsample adjusting for covariates), there is also a likelihood of over 95% of the CR + SE group being more cost-effective, with a threshold value of $30–$40 for a 1% increase in the number of people who work, and a threshold value of $20–$40 for an additional day of work tenure.
Cost-effectiveness acceptability curves for employment rate showing probabilities that cognitive remediation and supported employment (CR+SE) are more cost-effective than traditional vocational services.
Cost-effectiveness acceptability curves for job tenure showing probabilities that cognitive remediation and supported employment (CR+SE) are more cost-effective than traditional vocational services.
Cost-effectiveness acceptability curves for Brief Assessment of Cognition in Schizophrenia Japanese language version (BACS-J) one-unit improvement showing probabilities that cognitive remediation and supported employment (CR+SE) are more cost-effective than traditional vocational services.
The cost-effectiveness acceptability curve for the BACS-J illustrated a 71% likelihood of the CR + SE group being more cost-effective if people place a zero value on a unit improvement in the BACS-J composite score. These probabilities increased (78% with $0) after adjusting for covariates, and improved (89% with $0) in the schizophrenia subsample after adjusting for covariates. For all three curves, the probability of CR + SE being a more cost-effective option reaches 95% if society is willing to pay $300–$800 to improve cognitive function by 0.1 on the BACS-J composite score.
Discussion
Effects and costs of services
Although there was a difference in the employment rate between the sites, CR + SE appears overall to yield good vocational outcomes and financial benefits during 12-month follow-up in Japan even among people with mental illness and cognitive impairment. In particular, the TVS group had high costs of medical services that were obviously due to the cost of in-patient care. While employment can be an indicator of recovery (Thornicroft & Slade, Reference Thornicroft and Slade2014), it may be challenging for individuals with mental illness, particularly those with cognitive impairment (Tsang et al. Reference Tsang, Leung, Chung, Bell and Cheung2010). Past RCTs reported the effects of supported employment in preventing hospital admission (Heslin et al. Reference Heslin, Howard, Leese, McCrone, Rice, Jarrett, Spokes, Huxley and Thornicroft2011; Hoffmann et al. Reference Hoffmann, Jackel, Glauser, Mueser and Kupper2014). Likewise, this study showed that CR + SE, which in combination provided cognitive training, employment services and living support, also seemed to be effective in the prevention of hospitalization, thus reducing costs.
Regarding intervention costs in this study, the CR + SE approach would be more costly than TVS, although similar costs were reported when excluding the cognitive remediation costs. Presumably the cost of cognitive remediation in this study would be higher than in real-life settings. In this study, the participants received supported employment services after completion of cognitive remediation. This indicates that the subjects participated in almost all cognitive remediation sessions. However, it is expected that users in real-life settings concurrently receive cognitive remediation sessions and supported employment services to encourage early employment. Thus, some would be successfully employed before the completion of all cognitive remediation sessions and may be unable to attend psychiatric day-care or community employment service agency due to their new work obligations. This may lead to a reduction in the number of cognitive remediation sessions that some users receive, and accordingly to a lower cost of cognitive remediation in actual service settings. In fact, a past RCT found that the users who did not participate in all sessions of the cognitive rehabilitation and work programme had found new employment (McGurk et al. Reference McGurk, Mueser and Pascaris2005). However, it is also considered that individuals who do attend a number of cognitive remediation sessions are likely to demonstrate cognitive improvement (Medalia & Richardson, Reference Medalia and Richardson2005), which is associated with vocational outcomes (Tsang et al. Reference Tsang, Leung, Chung, Bell and Cheung2010). As Keefe et al. (Reference Keefe, Vinogradov, Medalia, Buckley, Caroff, D'Souza, Harvey, Graham, Hamer, Marder, Miller, Olson, Patel, Velligan, Walker, Haim and Stroup2012) pointed out, the ideal dose and frequency of cognitive remediation programmes are still debatable in real-life settings, and further research is needed to confirm the costs of providing concurrent CR + SE.
The site differences in employment rates in the CR + SE group may be attributed to variations in service quality of supported employment, rather than cognitive remediation, because there were no significant differences in cognitive improvement between the sites. Two recent studies revealed that when the quality of employment services provided after (or with) cognitive remediation was not high, such services did not produce good vocational outcomes, compared with the control group (Sato et al. Reference Sato, Iwata, Furukawa, Matsuda, Hatsuse and Ikebuchi2013; McGurk et al. Reference McGurk, Mueser, Xie, Feldman, Shaya, Klein and Wolfe2016). Although we provided the short-term training for staff in the CR + SE group, this may have been insufficient to result in effective SE at all sites.
Acceptability
The cost-effective acceptability curves suggest that there are relatively high probabilities (over 70%) of CR + SE being a cost-effective option for vocational outcomes and cognitive function, even if policy makers initiate no additional budget increases. Additionally, these probabilities rise to over 95% if policy makers are willing to pay less than $40 for an additional 1% of people working and an additional day of work tenure, and $800 for improvement in cognitive function for a year even after adjusting for potential confounders. These data indicate that CR + SE is affordable, even in a Japanese setting.
The probabilities for each outcome in the schizophrenia subsample were slightly improved compared with the whole sample when the values of society's willingness to pay were zero. Cognitive remediation has been developed mainly to target individuals with schizophrenia (Chan et al. Reference Chan, Hirai and Tsoi2015). In addition, cognitive impairment associated with negative symptoms in schizophrenia is likely to be more severe than in other disorders (Bora et al. Reference Bora, Yucel and Pantelis2009). Bell et al. (Reference Bell, Choi, Dyer and Wexler2014) also found that CR + SE was particularly effective in people with schizophrenia and low quality of life-based community functioning. Although we did not assess quality of life and found that the symptom severity (PANSS score) in this study was lower than that in Bell's study, people with schizophrenia are in general likely to have lower community functioning (Bellack et al. Reference Bellack, Green, Cook, Fenton, Harvey, Heaton, Laughren, Leon, Mayo, Patrick, Patterson, Rose, Stover and Wykes2007). These facts suggest that CR + SE may be particularly cost-effective among people with schizophrenia.
Comparison with past studies
Since this study did not have a control group with participants who underwent either only cognitive remediation or only supported employment, it is important to compare the present study with past findings. The average employment rate among those receiving supported employment was 56% according to a systematic review (Bond et al. Reference Bond, Drake and Becker2012). Adding cognitive remediation to supported employment was found to be particularly effective in individuals with mental illness who failed to respond to supported employment (McGurk et al. Reference McGurk, Mueser and Pascaris2005, Reference McGurk, Mueser, Xie, Welsh, Kaiser, Drake, Becker, Bailey, Fraser, Wolfe and McHugo2015). Indeed, this combination programme yielded high employment rates (over 60%) despite the challenges faced by this population. However, these international findings do not always apply to Japanese settings. A past RCT in Japan which assessed supported employment in a community sample demonstrated a reasonably high employment rate (44%) (Oshima et al. Reference Oshima, Sono, Bond, Nishio and Ito2014), but another RCT for an assertive community treatment sample reported a low employment rate (23%) at 12-month follow-up (Ogawa et al. Reference Ogawa, Nishio, Tsuda, Ishii, Koda, Hisanaga and Ito2008). In addition, a previous trial based in Japan which compared CR + SE with only employment services in people with schizophrenia and cognitive impairment reported no differences in vocational outcomes between the groups (Sato et al. Reference Sato, Iwata, Furukawa, Matsuda, Hatsuse and Ikebuchi2013). This suggests that at least in Japan, the use of only supported employment or cognitive remediation has limited effect in people with severe mental illness who probably have cognitive impairment. In other words, the better vocational outcomes observed in this study compared with past Japanese studies are probably due to the interaction between cognitive remediation and supported employment.
With regards to economic benefit, mental health costs and the cost of supported employment vary considerably between countries (Clark et al. Reference Clark, Xie, Becker and Drake1998; Dixon et al. Reference Dixon, Hoch, Clark, Bebout, Drake, Mchugo and Becker2002; Patel et al. Reference Patel, Knapp, Romeo, Reeder, Matthiasson, Everitt and Wykes2010; Heslin et al. Reference Heslin, Howard, Leese, McCrone, Rice, Jarrett, Spokes, Huxley and Thornicroft2011; Knapp et al. Reference Knapp, Patel, Curran, Latimer, Catty, Becker, Drake, Fioritti, Kilian, Lauber, Rossler, Tomov, Van Busschbach, Comas-Herrera, White, Wiersma and Burns2013; Hoffmann et al. Reference Hoffmann, Jackel, Glauser, Mueser and Kupper2014; Reeder et al. Reference Reeder, Harris, Pickles, Patel, Cella and Wykes2014). The generalizability of the findings in this study may be limited.
Limitations
The study has limitations. First, our sample size was calculated based on employment rates from a past RCT, and may be small for economic evaluation. This is particularly the case for the schizophrenia sample, since cost-effectiveness analysis with bootstrap methods is ideally suitable when the sample size is over 50 and the data have low skewness (Nixon et al. Reference Nixon, Wonderling and Grieve2010).
Second, no masked assessment was conducted for outcomes and costs, excluding medical costs; therefore, there may be an assessor bias. Instead, we could have ideally obtained complete employment outcomes and cost data for most participants, since individuals’ care managers conducted their interviews to collect these data. It is noted that cost data reliability was better when data were collected by professionals familiar with individuals and their community services (Mirandola et al. Reference Mirandola, Bisoffi, Bonizzato and Amaddeo1999).
Third, we did not assess the costs of training CR + SE staff. The training costs may be slightly underestimated, but the cost of this short training did not appear to substantially affect the total cost. In addition, we did not collect pre-randomization costs. It was impossible to control the influence of individual pre-randomization costs on the total costs in the economic analyses, although we used the first month's cost as an alternate variable to control the influence of the cost of continuous service use, before and after study participation, on the total cost.
Fourth, the current study did not employ any measures for quality-adjusted life years. These limitations should be addressed in the future to make progress in this area.
Implications for policy, practice and future research
Employment of people with mental illness is a major interest of not only users and service providers but also policy makers. This study showed that CR + SE resulted in better vocational and cognitive outcomes in people with mental illness, with affordable costs compared with TVS in Japan. In particular, CR + SE appears to be a cost-effective service for people with schizophrenia. However, this study did not compare CR + SE in terms of which provided greater clinical and economic benefits. Clarifying this issue should be a goal of future studies.
Supplementary material
The supplementary material for this article can be found at http://dx.doi.org/10.1017/S0033291716002063
Acknowledgements
This work was supported by the Ministry of Health, Labour and Welfare in Japan (H23–001: A study on the model construction and the evaluation of community psychiatry to promote community lives). We would like to thank all the research and clinical collaborators: Miho Furuie, Yasuhiro Hagiwara, Ken Ichikawa, Dr Sachie Inoue, Kazuko Ishi, Nobuyuki Izumida, Dr Makoto Kobayashi, Dr Masuhiro Sakata, Professor Martin Knapp, Dr Naoko Satake, Yuta Taguchi, Noriko Umeda and Takuya Usui.
Declaration of Interest
None.
