INTRODUCTION
Approximately 795,000 Americans sustain a stroke each year and require rehabilitation services (physical therapy, occupational therapy, speech therapy) to regain function and return to the community (Mozaffarian et al., Reference Mozaffarian, Benjamin, Go, Arnett, Blaha, Cushman and Turner2016). Promoting high engagement during inpatient rehabilitation after stroke may support optimal rehabilitation outcomes (e.g., disability status, length of stay, discharge disposition). Low client engagement in inpatient rehabilitation is associated with long length of stay, discharge disposition, and poor functional outcomes (Lenze et al., Reference Lenze, Munin, Quear, Dew, Rogers, Begley and Reynolds2004a; Paolucci et al., Reference Paolucci, Di Vita, Massicci, Traballesi, Bureca, Matano and Guariglia2012). In medical rehabilitation, client engagement is often defined by observable behaviors produced by clients that indicate active interest and involvement in the rehabilitation process (Bright, Kayes, Worrall, & McPherson, Reference Bright, Kayes, Worrall and McPherson2015).
Engagement in rehabilitation exists along a continuum (Matthews et al., Reference Matthews, Campbell, Falconer, Joyner, Huggins, Gilliland and Warm2002). Clients who have high engagement demonstrate active interest by asking questions and providing input to the therapist during therapy tasks. They put forth full effort and complete all therapy tasks. Clients who have low engagement demonstrate apathy or require encouragement to complete therapy tasks. They do not complete all tasks during therapy sessions and do not ask questions or provide input during therapy (Lequerica, Donnell, & Tate, Reference Lequerica, Donnell and Tate2009). Identifying specific clinical variables that predict engagement during post-stroke rehabilitation may afford clinical teams an opportunity to identify clients who are at risk for low engagement and, therefore, poor functional outcomes. If clinical teams can identify clients who are at risk for low engagement, they may be able to implement strategies to promote engagement during therapy sessions that can lead to positive functional outcomes.
Available tools that measure engagement examine client attendance, task completion, effort expended, and interest in the therapy session to determine the level of engagement in medical rehabilitation (Kortte, Falk, Castillo, Johnson-Greene, & Webener, Reference Kortte, Falk, Castillo, Johnson-Greene and Wegener2007; Lenze et al., Reference Lenze, Munin, Quear, Dew, Rogers, Begley and Reynolds2004b). Engagement during inpatient occupational therapy (OT) and physical therapy (PT) sessions has been measured in rehabilitation populations using the Pittsburgh Rehabilitation Participation Scale (Lenze et al., Reference Lenze, Munin, Quear, Dew, Rogers, Begley and Reynolds2004b). Using this tool, low engagement has been detected in 21–33% of rehabilitation samples, including clients with neurologic, orthopedic, and general debility diagnoses (Lenze et al., Reference Lenze, Munin, Quear, Dew, Rogers, Begley and Reynolds2004a; Paolucci et al., Reference Paolucci, Di Vita, Massicci, Traballesi, Bureca, Matano and Guariglia2012).
The Pittsburgh Rehabilitation Participation Scale requires clinicians to classify clients’ engagement-related behaviors on a scale of 1 to 6 (Lenze et al., Reference Lenze, Munin, Quear, Dew, Rogers, Begley and Reynolds2004b). To provide a single score, clinicians consider the client’s task completion, effort, and perceived interest in the therapy session. Using this scale, previous studies identified that mood, cognition, and disability at admission were associated with engagement in general rehabilitation samples (Lenze et al., Reference Lenze, Munin, Quear, Dew, Rogers, Begley and Reynolds2004a; Paolucci et al., Reference Paolucci, Di Vita, Massicci, Traballesi, Bureca, Matano and Guariglia2012; Skidmore et al., Reference Skidmore, Whyte, Holm, Becker, Butters, Dew and Lenze2010). Stroke usually culminates in significant changes in mood and cognition as well as pronounced disability at admission to inpatient rehabilitation, placing individuals with stroke at risk for low engagement and poor functional outcomes.
One previous study examined engagement in inpatient rehabilitation following acute stroke (Skidmore et al., Reference Skidmore, Whyte, Holm, Becker, Butters, Dew and Lenze2010). This study, that included individuals who received pharmacological intervention for cognitive impairments, found that executive function was the strongest predictor of engagement in rehabilitation following stroke. However, the study only examined cognitive and affective predictors of engagement. To extend this work, we aimed to conduct a secondary analysis that explored a wider range of clinical factors using more precise measures (i.e., specific cognitive domains, affect, motor function) to identify clinical factors that could distinguish individuals likely to have high engagement from those likely to have low engagement in rehabilitation following stroke. Based on previous studies (including our own), we anticipated that executive functions would be a strong predictor of the level of engagement in inpatient rehabilitation following stroke after controlling for baseline disability (Lenze et al., Reference Lenze, Munin, Quear, Dew, Rogers, Begley and Reynolds2004a; Paolucci et al., Reference Paolucci, Di Vita, Massicci, Traballesi, Bureca, Matano and Guariglia2012; Skidmore et al., Reference Skidmore, Whyte, Holm, Becker, Butters, Dew and Lenze2010).
METHODS
Participants
Data were compiled from three studies registered through ClinicalTrials.gov (NCT02755805, NCT02766400, NCT01934621) that examined the feasibility and efficacy of strategy training in adults with cognitive deficits following acute stroke during inpatient rehabilitation. In the parent studies, participants were recruited from six inpatient rehabilitation facilities within a single university healthcare system. Participants were included in the parent studies if they had a diagnosis of stroke, were admitted to inpatient rehabilitation, and scored≥3 on the Quick-EXIT (Larson & Heinemann, Reference Larson and Heinemann2009). The Quick-EXIT was used to ensure that all participants had cognitive deficits (could be mild to severe), which was a requirement for the parent studies. Participants were excluded if they had premorbid dementia (documented in medical record), severe aphasia (≤1 on the Boston Diagnostic Aphasia Examination severity scale, BDAE, 3rd Edition; Borod, Goodglass, & Kaplan, Reference Borod, Goodglass and Kaplan1980), current major depressive disorder (Patient Health Questionnaire-9; Gilbody, Richards, Brealey, & Hewitt, Reference Gilbody, Richards, Brealey and Hewitt2007; Spitzer, Kroenke, & Williams, Reference Spitzer, Kroenke and Williams1999), current bipolar or psychotic disorder (PRIME-MD; Spitzer et al., Reference Spitzer, Williams, Kroenke, Linzer, deGruy, Hahn and Johnson1994), or active alcohol or substance abuse within the prior 3 months (PRIME-MD; Spitzer et al., Reference Spitzer, Williams, Kroenke, Linzer, deGruy, Hahn and Johnson1994).
Interventions
Participants in the parent studies were randomized to receive either strategy training, attention control, or direct skill training control intervention. They received one study intervention session per day in addition to standard rehabilitation care. Participants received 10 sessions of the study intervention during their stay in inpatient rehabilitation. Study interventions are described in previous literature (Skidmore et al., Reference Skidmore, Dawson, Butters, Grattan, Juengst, Whyte and Becker2015, Reference Skidmore, Butters, Whyte, Grattan, Shen and Terhorst2017). Briefly, strategy training is a meta-cognitive approach that aims to teach clients skills to achieve their personal goals using a global strategy (Goal-Plan-Do-Check). Guided discovery is used during strategy training to help clients identify solutions to any challenges encountered during their goals (Skidmore et al., Reference Skidmore, Dawson, Butters, Grattan, Juengst, Whyte and Becker2015, Reference Skidmore, Butters, Whyte, Grattan, Shen and Terhorst2017).
Two of the parent studies contrasted strategy training with a control emotional support intervention, in which the therapist met with participants to reflect on their rehabilitation process. Participants set personal goals, but the therapist did not offer strategies for achieving these goals. One of the parent studies contrasted strategy training with direct skill training, in which the therapist selected the goal and task during each intervention session, reflective of usual care rehabilitation. For the present analysis, participants in all groups (strategy training, attention control, and direct skill training) were analyzed as one cohort. The University Institutional Review Board approved all study procedures and informed consent was obtained from study participants.
Primary Outcome Measure
The Pittsburgh Rehabilitation Participation Scale (PRPS) was administered by staff occupational and physical therapists for every therapy session conducted during standard inpatient rehabilitation. The PRPS is a 6-point Likert scale where 1 indicates poor engagement and 6 indicates excellent engagement (Lenze et al., Reference Lenze, Munin, Quear, Dew, Rogers, Begley and Reynolds2004b). Scores were assigned based on the treating therapist’s observations of the participant’s task completion, effort, and active interest. The PRPS was completed for each standard occupational and physical therapy session during 10 days of rehabilitation. Authors of the PRPS describe sessions scored as a 4 in which the individual completes most but not all of the therapy activities, and gives good effort but not maximal effort. Participants who participate in sessions in this manner passively follow directions but do not ask additional questions or take active interest in the session.
Sessions scored as 5 or 6 are indicative of a highly engaged participant, in which maximal effort is given and all exercises are completed. Clients take active interest in therapy during these sessions, which is indicated by asking questions and giving input during the therapy session. If reasons for not attending therapy sessions were directly related to medical issues (i.e., bedrest due to deep vein thrombosis), raters were instructed to not score the session. Raters were provided with brief written descriptors of each score. Using this training method, the PRPS demonstrates high interrater reliability (ICC=.91 for occupational therapists, ICC=.96 for physical therapists; Lenze et al., Reference Lenze, Munin, Quear, Dew, Rogers, Begley and Reynolds2004b).
Clinical Predictors
Possible clinical predictors of engagement included cognition (language, visuospatial skills, attention, immediate memory, delayed memory, executive functions), mood, and motor function. All measures of possible clinical predictors were collected on admission to rehabilitation. We dichotomized each measure to identify participants as impaired and not impaired for each domain. Norm-referenced measures were dichotomized with clients who scored 1 standard deviation or greater below the mean coded as impaired, and all others as not impaired. Criterion-referenced scales that had previously established classifications for impairment were dichotomized using these reference points. We calculated a median split for any criterion-referenced scales that did not have previously established classifications for severity of impairment. Specific dichotomization strategies for each domain are described in greater detail in the following sections.
Cognition
Language, visuospatial skills, attention, immediate memory, and delayed memory were measured using the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). The RBANS consists of 12 subtests that are used to calculate index scores for each of the five domains. Age-adjusted index scores have a mean of 100 with a standard deviation of 15. Index scores for each domain were dichotomized to identify participants with impairments in each domain. Participants who scored greater than one standard deviation below the mean (RBANS Index Score ≤85) were identified to have impairments in that domain of cognition (0=intact; 1=impaired). The RBANS has been well validated with high sensitivity to characterize cognitive deficits in the general adult population and in the stroke population (Larson, Kirschner, Bode, Heinemann, & Goodman, Reference Larson, Kirschner, Bode, Heinemann and Goodman2005; Randolph, Tierney, Mohr, & Chase, Reference Randolph, Tierney, Mohr and Chase1998). The RBANS provides a comprehensive assessment of specific cognitive domains, but does not measure executive functions.
We measured executive functions via select subtests of the Delis-Kaplan Executive Function System (D-KEFS). The D-KEFS is a measure that demonstrates reasonable reliability, validity, and sensitivity for measuring the complexities of executive functions (Delis, Kramer, Kaplan, & Holdnack, Reference Delis, Kramer, Kaplan and Holdnack.2004; Homack, Lee, & Riccio, Reference Homack, Lee and Riccio2005). We derived a single score for executive functions by combining the D-KEFS Verbal Fluency Conditions 1 (phonemic) and 2 (semantic) with the D-KEFS Trail Making Test Condition 4. We acknowledge that, in some cases, verbal fluency is considered to be more of a language than an executive function based task (e.g., Whiteside et al., Reference Whiteside, Kealey, Semla, Luu, Rice, Basso and Roper2016).
However, in addition to language, tests of verbal fluency require multiple executive functions including self-monitoring, working memory, suppression of prior responses, and cue generation (Rosen & Engle, Reference Rosen and Engle1997). Due to these task requirements and factor analytic findings placing verbal fluency and tests of executive function within the same factor, we elected to combine verbal fluency and switching to derive an overall executive function score (Elias et al., Reference Elias, Elias, D’Agostino, Silbershatz and Wolf1997). We first converted individual raw scores to age-adjusted standardized scores with a mean of 10 and standard deviation of 3. We then calculated the mean of Verbal Fluency Condition 1 and Condition 2 to create a single Verbal Fluency score. Finally, we calculated the mean of Verbal Fluency and Trail Making Test Condition 4 to derive a single score for executive functions. These age-adjusted standardized scores have a mean of 10 and standard deviation of 3. Participants who scored greater than one standard deviation below the mean (D-KEFs score ≤7) were identified as having impairments in executive functions.
Mood
Mood was measured using the Patient Health Questionnaire-9 (PHQ-9). Possible scores on the PHQ-9 range from 0 to 27, where high scores indicate more severe depressive symptoms. We identified participants with minimal depressive symptoms (PHQ-9 score ≤5, coded 0) and those with mild or greater depressive symptoms (PHQ-9 score ≥6, coded 1). The PHQ-9 demonstrates acceptable reliability and validity (Gilbody et al., Reference Gilbody, Richards, Brealey and Hewitt2007; Spitzer et al., Reference Spitzer, Kroenke and Williams1999).
Motor function
Motor function was measured using the Chedoke McMaster Stroke Assessment (CMA). The CMA measures motor recovery as it relates to stages of recovery following stroke. Six items are scored and totaled, ranging from 0 to 42. Low scores indicate worse motor function. We distinguished those in our sample with low and high levels of motor impairment using a median split (CMA scores≥26=low impairment, coded 0; CMA scores ≤25=high impairment, coded 1). The CMA has good reliability and validity in the stroke population (Gowland et al., Reference Gowland, Stratford, Ward, Moreland, Torresin, Van Hullenaar and Plews1993).
Covariates
The purpose of this study was to identify which impairments in specific clinical domains most strongly predicted low engagement. We controlled for baseline stroke severity and baseline disability because these are overall measures of neurological impairment and baseline disability that are not domain specific. These two broad constructs combine multiple domains into single ratings of impairments at the body functions (stroke severity) and daily life task ability (disability) levels of function. We also examined age as a potential covariate. Age was excluded from the model as the association was not significant (p=.426).
Stroke severity
Stroke severity was measured using the National Institutes of Health Stroke Scale (NIHSS). Possible scores on the NIHSS range from 0 to 42, where high scores indicate high stroke severity. The NIHSS demonstrates moderate to substantial interrater reliability and is well validated within the stroke population (Goldstein, Bertels, & Davis, Reference Goldstein, Bertels and Davis1989; Goldstein & Samsa, Reference Goldstein and Samsa1997; Lyden et al., Reference Lyden, Lu, Jackson, Marler, Kothari, Brott and Zivin1999). We used the baseline NIHSS scores to identify participants with mild stroke (NIHSS≤5, coded 0) and moderate to severe stroke (NIHSS≥6, coded 1). These scores were used to control for stroke severity in the model based on previously established associations between stroke severity and rehabilitation outcomes (Jorgensen et al., Reference Jorgensen, Nakayama, Raaschou, Vive-Larsen, Stoier and Olsen1995).
Disability
Disability was measured using the Functional Independence Measure (FIM). Possible scores on the FIM range from 18 to 126, where high scores indicate low disability. The FIM demonstrates strong reliability and validity within rehabilitation populations (Hsueh, Lin, Jeng, & Hsieh, Reference Hsueh, Lin, Jeng and Hsieh2002; Ottenbacher, Hsu, Granger, & Fiedler, Reference Ottenbacher, Hsu, Granger and Fiedler1996; Stineman et al., Reference Stineman, Shea, Jette., Tassoni, Ottenbacher, Fiedler and Granger1996). Disability is strongly associated with rehabilitation outcomes and was previously identified as a meaningful predictor of engagement in inpatient rehabilitation (Lenze et al., Reference Lenze, Munin, Quear, Dew, Rogers, Begley and Reynolds2004a; Paolucci et al., Reference Paolucci, Di Vita, Massicci, Traballesi, Bureca, Matano and Guariglia2012; Ween, Alexander, D’Esposito, & Roberts, Reference Ween, Alexander, D’Esposito and Roberts1996). The FIM is an omnibus measure of disability that combines physical and cognitive factors into one score. We distinguished those in our sample with low and high levels of disability using a median split (FIM scores≥89=low impairment, coded 0; FIM scores ≤88=high impairment, coded 1). We used these scores to control for baseline levels of overall disability in the analysis.
Study intervention group
The data for the current study were compiled from three prior studies in which participants received either strategy training, or a control intervention (attention control or direct skill training). To control for potential effects of the interventions in the parent studies, we created one dummy variable to represent the experimental intervention and the control group intervention (Strategy Training, Control Intervention). Because both control group conditions were reflective of standard therapy approaches and did not contain the active ingredients of the experimental intervention, participants who received either the attention control or direct skill training control were grouped together and were coded as Control Intervention. These variables were forced into the model to control for study intervention group.
Statistical Analyses
We examined histograms of engagement scores for 10 days of inpatient rehabilitation. First, we calculated daily mean engagement scores (including both PT and OT sessions). Using SAS® software (2012, Cary, NC), we then examined plots of daily mean engagement scores for individuals over 10 days of rehabilitation. The plots allowed for visual inspection of within-individual engagement trajectories as well as between-individual variability and overall “shape” or pattern of engagement. These observations of low within-individual variability guided our decision to dichotomize engagement outcomes to identify those with high engagement and low engagement.
We used a theoretical approach to dichotomize high and low engagement based on descriptors of the scale and conceptualization of engagement described in the literature (Bright et al., Reference Bright, Kayes, Worrall and McPherson2015; Lenze et al., Reference Lenze, Munin, Quear, Dew, Rogers, Begley and Reynolds2004b; Lequerica et al., Reference Lequerica, Donnell and Tate2009). A PRPS score equal to 4 indicates that the client passively followed directions and completed some, but not all, therapy activities. Scores less than 4 indicate that clients did not complete therapy activities or did not give full effort during therapy tasks. Clients who scored less than or equal to 4 for 10 or more therapy sessions (of a possible 40) demonstrated a pattern of passively following directions and completing some, but not all, therapy activities. They were identified to have low engagement. All others were identified to have high engagement. We examined mean differences on baseline characteristics for those with low and high engagement using independent samples t test or chi-square test, as appropriate.
Logistic regression analyses were conducted with IBM SPSS Statistics for Windows (Version 23.0. Armonk, NY:IBM Corp) and analyzed against a criterion of α=.10. We selected this criterion for this exploratory analysis so that potentially relevant predictors were not overlooked. We first conducted simple logistic regression with separate models for each clinical predictor to examine associations with level of engagement. Clinical predictors that were statistically significant (p<.10) in the simple logistic regression models were included in a hierarchical logistic regression to identify the strongest predictors of engagement. Study intervention group, stroke severity, and disability were force-entered in step 1 to control for effects of the interventions that were administered in the parent studies and the effects of stroke severity and disability. To determine if more precise clinical variables could better classify level of engagement, potential predictors were entered in step 2 and analyzed using backward stepwise regression.
The criterion for removal from the model was p<.10; thus, if predictors did not contribute to the model they were removed and a final, best-fitting model was selected. It is important to note that the covariates (study intervention group, baseline stroke severity, and baseline disability) were forced into the model; therefore, they appear in both step 1 and step 2. Model fit was assessed using deviance testing, to determine if the addition of clinical predictors in step 2 demonstrated a significant improvement over the covariates that were force-entered in step 1 (Field, Reference Field2013). Odds ratios (ORs) of individual variables that remained in the final, best-fitting model were examined. Predictors with a moderate or greater OR (0–2.5=small; 2.6–3.5=moderate; 3.5 or higher=large) were considered meaningful for predicting individuals who had high or low engagement (Chen, Cohen, & Chen, Reference Chen, Cohen and Chen2010).
Missing data
All participants without missing data on the individual predictor were included in the simple logistic regressions. Those who were unable to complete tests of a particular domain due to impairments in that domain (e.g., unable to complete cognitive tests due to cognitive impairment or unable to complete visuospatial tests due to neglect) were coded as impaired and included in the regression analysis. Those who were unable to complete tests in particular domains due to unrelated impairments (e.g., unable to complete paper and pencil tests of cognition due to motor impairments) or for other reasons (e.g., study withdrawal) were excluded from the analysis. Those with missing data on one or more variables were excluded from the hierarchical regression analysis. Specific reasons for missing data from the 33 participants who were excluded from the hierarchical regression analysis are identified in results.
RESULTS
Participant Characteristics
The combined sample of the parent studies consisted of 208 participants with stroke as the primary reason for admission to six different university associated hospital-based inpatient rehabilitation facilities. Thirty-three participants were excluded from the hierarchical regression analysis due to: vision impairments (11), physical impairments (4), study withdrawal or refusal to complete testing (14), and reasons unrelated to the client (e.g., time constraints during testing, 4). Table 1 describes the sample included in our hierarchical regression analysis (n=175). Participants were, on average, 67.47 years of age, and 50.3% were male. The majority of the sample sustained ischemic stroke (76.6%). Sub-cortical stroke was the most common (52.0%) followed by cortical (38.3%) and both cortical and subcortical (9.7%). Right hemisphere (48.6%) and left hemisphere (42.9%) lesions were approximately the same proportion.
Baseline characteristics
a Mean (SD) or n (percentage).
b Chronicity: Days elapsed from date of stroke diagnosis to inpatient rehabilitation admission.
c Executive functions score calculated using DKEFS Verbal Fluency Condition 1, DKEFS Verbal Fluency Condition 2, and DKEFS Trail Making Test Condition 4.
*Statistically significant at p<.05.
Participants received strategy training (48.0%) or the control intervention (52.0%) in addition to standard rehabilitation care. All participants had cognitive deficits on admission to acute rehabilitation identified by scores on the Quick-EXIT (M=9.45). Participants had slight or no aphasia, measured by the BDAE severity scale (M=4.64). During 10 days of rehabilitation, participants engaged in an average of 13.05 standard occupational and physical therapy sessions combined. Within the sample, 26.8% had low engagement and 73.1% had high engagement.
Figure 1 represents the daily mean engagement trajectories of the sample. Our visual inspection showed low within-individual variability over the 10 days; however, we did see an overall pattern that showed a range of low to high engagement between individuals. Based on the trajectory plots and prior research, we dichotomized participants to have high (0) and low (1) engagement. Group differences on baseline characteristics between those with high and low engagement are reported in Table 1.
Spaghetti plot of daily mean scores on Pittsburgh Rehabilitation Participation Scale over 10 days of inpatient rehabilitation following stroke. The x-axis represents days 1 through 10 of inpatient rehabilitation. Scores on the y-axis indicate daily mean ratings of combined occupational therapy and physical therapy sessions.
Predictors of Engagement
Simple logistic regression
Simple logistic regression analyses were conducted with level of engagement (high, low) as the dependent variable and individual clinical predictors as the independent variables (Table 2). Intervention group, stroke severity, disability, gender, language skills, immediate memory, delayed memory, visuospatial skills, and executive functions, mood, and motor function were statistically significant individual predictors of level engagement (p<.10). These variables were retained for hierarchical logistic regression. We excluded length of stay, chronicity, and number of therapy sessions attended from the hierarchical logistic regression model to avoid multicollinearity.
Simple logistic regression
a Variables retained for hierarchical regression analysis.
b Group assignment relative to Strategy Training Group.
c Executive Functions score calculated using DKEFS Verbal Fluency Condition 1, DKEFS Verbal Fluency Condition 2, and DKEFS Trail Making Test Condition 4.
Hierarchical logistic regression
A hierarchical logistic regression analysis was conducted using two blocks (Table 3). Study intervention group, stroke severity, and disability were forced into block 1, and this model was an improvement over the intercept-only model, (χ2(3)=15.287; p=.002), indicating that the addition of these covariates was important to classify individuals’ level of engagement. Disability was the only individually reliable predictor of the level of engagement in block 1 (OR=2.754; 95% CI 1.295, 5.856; p=.009).
Backward stepwise logistic regression
Note. Predictors that are shaded in Block 2 were forced into the model. Unshaded items are clinical predictors of interest.
a Group assignment relative to Strategy Training Group.
b Block 2 includes only variables remaining in final model after backward stepwise regression completed.
c Executive Functions score calculated using DKEFS Verbal Fluency Condition 1, DKEFS Verbal Fluency Condition 2, and DKEFS Trail Making Test Condition 4.
d Male gender relative to female gender.
Clinical variables significantly associated with engagement in the simple logistic regression analyses (gender, language skills, immediate memory, delayed memory, visuospatial skills, executive functions, mood, motor function) were then added as potential predictors in block 2. Backward stepwise regression was used to remove predictors with p-values less than .10, yielding a final, best-fitting model. Study intervention group, baseline stroke severity, and baseline disability were forced into block 2. After the backward stepwise regression was completed, the clinical variables of gender, visuospatial skills (RBANS Visuospatial Index Score), executive functions (single score derived from D-KEFS Verbal Fluency Condition 1, Condition 2, and D-KEFS Trail Making Test Condition 4), and mood (Patient Health Questionnaire-9) were retained as predictors of level of engagement (Table 3). The model was reliable (χ2(7)=36.326; p<.001), and deviance testing revealed that the addition of clinical predictors in block 2 led to improved model fit over block 1 (p<.001).
Remaining mindful of the exploratory nature of this work, we considered the effect size of clinical variables that were retained in the final model with α=.10 (Moore, Carater, Nietert, & Stewart, Reference Moore, Carter, Nietert and Stewart2011). Impaired executive functions (OR=4.062; 95% CI=.866, 19.051; p=.075) and visuospatial skills (OR=3.940; 95% CI=1.317, 11.785; p=.014) demonstrated a large effect size. Male gender (OR=2.474; 95% CI=1.145, 5.374; p=.021) and mood (OR=2.059; 95% CI=.953, 4.449; p=.066) demonstrated a small effect size. Thus, our sample demonstrated that executive functions and visuospatial skills were considered meaningful clinical variables for predicting engagement in rehabilitation after controlling for study intervention group, baseline stroke severity, and baseline disability.
Gender and mood were retained in the model and had a small effect size; thus, these variables may also contribute to distinguishing those with high and low engagement in rehabilitation after stroke. Those with impaired executive functions had 4.062 greater odds of having low engagement relative to those with intact executive functions. Those with impaired visuospatial skills had 3.940 greater odds of having low engagement relative to those with intact visuospatial skills. Males had 2.474 times greater odds of having low engagement relative to females. Those with mild or greater depressive symptoms had 2.059 times greater odds of having low engagement relative to those with minimal depressive symptoms.
A post hoc power analysis using G*Power (Faul, Erdfelder, Buchner, & Lang, Reference Faul, Erdfelder, Buchner and Lang2009) indicated that the statistical power for this study was .63 for detecting a small OR; however, power exceeded .85 to detect a moderate to large odds ratio. Therefore, we had more than adequate power to detect a moderate to large odds ratio, but less than adequate power to detect a small odds ratio. We also examined the associations among predictors remaining in the final model to assess for multicollinearity. None of these associations exceeded r=0.22, indicating that our predictors had low associations and, therefore, low multicollinearity.
DISCUSSION
Our goal was to replicate and expand upon previous work to identify specific clinical variables that could distinguish those who demonstrate high versus low engagement in rehabilitation following stroke. Identifying clinical variables that can distinguish high and low engagement in rehabilitation may allow clinical teams to identify those at risk for low engagement and implement strategies that promote engagement early during a client’s rehabilitation stay. This could lead to positive functional outcomes. Our large sample size (n=175) allowed us to explore a broad range of clinical variables that are frequently impaired after stroke (motor function, cognitive functions, affect).
In addition, our data set contained precise measures of cognitive domains, which allowed us to examine a wide array of cognitive domains (language, visuospatial skills, attention, immediate memory, delayed memory, executive functions). We controlled for baseline stroke severity and baseline disability because the NIHSS and FIM, our stroke severity and disability measures, are broad measures that encompass physical, cognitive, and affective deficits. The purpose of this study was to identify if clinical variables that measure precise deficits could predict low engagement.
Examination of additional clinical variables revealed that executive functions, visuospatial skills, gender, and mood may be meaningful predictors of high and low engagement during inpatient rehabilitation after stroke. Executive functions had the largest effect size, indicating that those with impaired executive functions had 4.062 greater odds of having low engagement relative to those with intact executive functions. This finding is consistent with previous work that identified executive functions as a meaningful predictor of engagement in rehabilitation after stroke (Skidmore et al., Reference Skidmore, Whyte, Holm, Becker, Butters, Dew and Lenze2010). Executive functions comprise the cognitive domains that allow clients to persist in goal-oriented behaviors, switch between tasks, inhibit learned behaviors while learning new behaviors, and use working memory within problem solving (Birn et al., Reference Birn, Kenworthy, Case, Caravella, Jones, Bandettini and Martin2010; Latzman & Markon, Reference Latzman and Markon2010; Stuss et al., Reference Stuss, Alexander, Hamer, Palumbo, Dempster, Binns and Izukawa1998; Troyer, Moscovitch, & Winocur, Reference Troyer, Moscovitch and Winocur1997; Zinn, Bosworth, Hoenig, & Swartzwelder, Reference Zinn, Bosworth, Hoenig and Swartzwelder2007).
Executive functions deficits are common in acute stroke and have been associated with poor rehabilitation outcomes. Inpatient rehabilitation after stroke frequently involves collaborating with the therapist to identify new strategies to safely accomplish previously learned tasks. This requires persistence in goal pursuits, inhibiting previously learned behaviors, and applying working memory to contribute to problem solving new strategies. If executive functions are impaired, a client whose neurological impairments preclude them from accomplishing these tasks may lead to an appearance of low engagement during therapy sessions.
In our analysis, visuospatial skills had a moderate effect size for predicting high and low engagement during inpatient rehabilitation. Lesions to neural networks that control attention and arousal may result in visuospatial neglect after stroke (Behrmann & Shomstein, Reference Behrmann and Shomstein2015; Boukrina & Barrett, Reference Boukrina and Barrett2017). Individuals with visuospatial neglect may demonstrate an inability to attend to one side of their environment and my also experience impairments in arousal and insight that impact the ability to attend to instructions during therapy sessions (Bourkina & Barrett, Reference Boukrina and Barrett2017; Nys et al., Reference Nys, van Zandvoort, de Kort, Jansen, de Haan and Kappelle2007).
Inpatient rehabilitation involves interacting with people and items in the client’s environment to regain daily living skills. For example, a therapist may provide verbal instructions and visual demonstration regarding safe strategies to get into the bathtub. A client who has visuospatial neglect may have difficulty attending to instructions and following through with the strategies that were demonstrated to safely complete the task. This may lead the therapist to conclude that the client has low engagement and is not invested in learning safe strategies for daily activities. It is possible that this client simply has difficulty attending to his/her environment during therapy sessions, requiring the therapist to approach teaching from a different perspective.
While gender and mood had a small effect for predicting high and low engagement during inpatient rehabilitation after stroke, we cannot ignore the potential contributions of these variables within the model. Males in our sample had 2.474 times greater odds of being perceived to have low engagement relative to females, which could be reflective of gender differences in relational or communication styles. In our sample, individuals with mild or greater depressive symptoms had 2.059 times greater odds of having low engagement that those with minimal depressive symptoms. Although the parent studies excluded participants who met diagnostic criteria for major depressive disorder, it was common for participants in these studies to experience depressive symptoms without meeting this criteria.
Depressive symptoms can occur during inpatient rehabilitation as clients adjust to new disability (Hackett, Yapa, Parag, & Anderson, Reference Hackett, Yapa, Parag and Anderson2005; Nys, Van Zandvoort, Vander Worp, De Haan, De Kort, & Kappele, Reference Nys, Van Zandvoort, Van der Worp, De Haan, De Kort and Kappelle2005). Depressive symptoms include sad mood, loss of interest in previously enjoyed activities, feelings of hopelessness, difficulty concentrating, and feeling tired, among others. Clients who are experiencing these symptoms may find it challenging to put forth full effort during therapy or actively participate in asking questions related to future goals. In addition, difficulty concentrating and feeling tired may be associated with difficulty completing all therapy tasks. Although depressive symptoms had a small effect size in our sample, it is reasonable to consider that mood may contribute to identifying those with low and high engagement in rehabilitation after stroke.
Engagement in rehabilitation was measured by the therapist’s observations of the client’s engagement-related behaviors (Kortte et al., Reference Kortte, Falk, Castillo, Johnson-Greene and Wegener2007; Lezne et al., 2004b). This type of measurement puts the responsibility of engagement on the client (Bright et al., Reference Bright, Kayes, Worrall and McPherson2015). We must consider that the therapist carries responsibility for client engagement as well. The manner with which the therapist interacts with the client may influence perceived client engagement. It is important for the clinical team to consider cognitive and mood impairments when assessing reasons for low engagement in rehabilitation. Perhaps a client who appears to have low engagement is highly motivated to work with the therapists but is unable to participate in problem solving or to follow through with completion of therapy tasks because of impaired executive functions or depressive symptoms. Perhaps a client who requires constant cues for safe transfer strategies appears to have low engagement, but instead was unable to attend to both fields of the environment due to neglect.
We speculate that increased awareness of the associations between executive functions, visuospatial skills, mood, and levels of engagement during rehabilitation may lead clinical teams to identify those who are at risk for low engagement and adapt service delivery for those with impairments in these areas. This may lead to increased engagement in therapy and improved functional outcomes of rehabilitation.
We also speculate that additional environmental and social factors which were not part of the available data set may influence perceived engagement in therapy. System-level factors that affect therapists (i.e., productivity demands, work schedules) may influence the manner in which goals are set and pursued (Levack, Dean, Siegert, & McPherson, Reference Levack, Dean, Siegert and McPherson2011; Wressle, Oberg, & Henriksson, Reference Wressle, Oberg and Henriksson1999). Client engagement may vary based on involvement in goal-setting and the specific goals that are pursued during therapy sessions.
In addition, routines of the rehabilitation unit may influence the amount of control that a client perceives over their situation. Some rehabilitation units offer more structured activity time during the day while others are more loosely structured. Client engagement may be influenced by their perceived level of control over their daily routines or schedule. Units that are more structured may also facilitate increased social interaction with staff or other clients (Lincoln, Willis, Philips, Juby, & Berman, Reference Lincoln, Willis, Philips, Juby and Berman1996). Additional client-specific factors may also influence the perceived level of client engagement. Social support from friends and family, personality, treatment expectancy, and level of fatigue are just a few additional factors that may influence client engagement.
Study Limitations
Although impaired executive functions, impaired visuospatial skills, impaired mood, and male gender were statistically significant independent predictors of low engagement after controlling for intervention group, baseline stroke severity, and disability, these results should be interpreted with caution. The confidence intervals for these predictors were wide, indicating variance in the sample. Due to the exploratory nature of this work, we prioritized the avoidance of a type II error, but remain mindful that risk for type I error is present (Jaeger & Halliday, Reference Jaeger and Halliday1998). Larger, more confirmatory studies may provide additional evidence that executive functions, visuospatial skills, and mood are important predictors of low engagement.
The method by which engagement was measured, the PRPS, is a limitation of this analysis. First, the PRPS only considers the clinician’s observation of the client’s behavior. Clients who cannot engage due to stroke-related impairments (e.g., executive functions, visuospatial skills, depressive symptoms) would be identified to have low engagement based on this scale. This highlights the importance of considering the client’s perspective on his/her level of engagement as tools to better measure engagement in rehabilitation are developed. Understanding the client’s perspective may aid clinicians in adapting their approach to rehabilitation, thus promoting engagement and optimal outcomes.
In addition, the client’s preferred type of or level of engagement is not considered within the PRPS. The influence of the relationship between the client and the clinician is not considered when only the client’s observable behaviors are considered. While tools exist for measuring the therapeutic alliance in psychotherapy and provider-patient relationship in medicine, tools designed to measure the dyadic nature of engagement in rehabilitation do not exist. This is an area for further research and development. Second, the PRPS combines the constructs of attendance, effort, and interest into one score. The clinician uses skilled observation to make a subjective determination of the client’s effort and interest. Subjectively combining these constructs into one score limits the specificity of the tool.
Selection bias can be problematic with secondary data analysis. Participants were only included in the parent studies if they demonstrated at least mild executive function impairments (Quick-EXIT score≥3). In addition, participants who demonstrated severe aphasia and mood disorders were excluded. While this sample represented participants with a range of cognitive abilities, results cannot be generalized to those without cognitive deficits. Individuals who have intact cognition following acute stroke may appear to engage differently than those with cognitive deficits. Aphasia and mood disorders are common following stroke. Due to the conversational nature of the interventions examined in the parent studies, those with aphasia were excluded from the sample. The exclusion of those with aphasia also allowed for usage of rigorous tests of cognitive function. This does not, however, allow generalization of these results to individuals with aphasia. We were also unable to identify if the presence or absence of communication deficits or mood disorders is predictive of engagement in rehabilitation.
Future Studies
To better understand the influence of executive functions, visuospatial skills, mood, and gender on engagement, it is important to characterize the interaction between the clinician and those clients who have impairments in these domains. This may involve analyses examining client-reported perspectives on the context of rehabilitation and the interaction between the clinician and the client. Understanding this context and the nature of these interactions may lead to greater insight on the measurement of this dyadic process.
In addition, other client-related variables (i.e., personality, motivation, fatigue, social support) may be associated with the level of engagement after stroke. The nature of secondary data analyses limited our ability to analyze these additional client variables. Understanding these additional variables may guide the identification of key elements that are important for promoting engagement in rehabilitation after stroke. Therefore, future research should consider the dyadic nature of engagement in rehabilitation, additional client-related variables, and the context within which rehabilitation occurs to better understand the interactions that influence engagement.
Approaches to promote optimal engagement in rehabilitation after stroke among clients who are identified to be at risk for low engagement should be explored. Patient-centered care and patient involvement in goal setting were identified as central components for promotion of patient engagement in rehabilitation settings (Cott, Wiles, & Devitt, Reference Cott, Wiles and Devitt2007; Holliday, Cano, Freeman, & Playford, Reference Holliday, Cano, Freeman and Playford2007; Lequerica et al., Reference Lequerica, Donnell and Tate2009). A variety of structured approaches to involve clients in goal-setting have been explored in general rehabilitation populations, with mixed effects on engagement and function-related outcomes (Levack et al., Reference Levack, Taylor, Siegert, Dean, McPherson and Weatherall2006; Rosewilliam, Roskell, & Pandyan, Reference Rosewilliam, Roskell and Pandyan2011; Stevens, Beurskens, Koke, & van der Weijden, Reference Stevens, Beurskens, Koke and van der Weijden2013; Sugavanam, Mead, Bulley, Donaghy, & van Wijck, Reference Sugavanam, Mead, Bulley, Donaghy and van Wijck2013; Watkins et al., Reference Watkins, Auton, Deans, Dickinson, Jack, Lightbody and Leathley2007). Clients with impairments in executive functions, visuospatial skills, and depressive symptoms may require additional support from the therapist to establish realistic goals, make connections between therapy activities that are conducted in an unfamiliar clinic environment to those activities that they desire to return to at home, and carry over skills between therapy sessions. Examination of variability in clients’ response to different goal-setting approaches based on impairments in executive functions, visuospatial skills, and depressive symptoms may inform adaptation of these approaches or development of new approaches for individuals with impairments in these areas. This holds promise to support the development of effective interventions for promoting engagement and, thereby, optimal outcomes of rehabilitation after stroke.
ACKNOWLEDMENTS
Conflict of Interest: The authors have no conflicts of interest to report. Funding: This work was supported by the National Institutes of Health (E.S., R03 HD073770, R01 HD074693); University of Pittsburgh Health Sciences; and UPMC Rehabilitation Institute. ClinicalTrials.gov Identification: NCT02755805, NCT02766400, NCT01934621.
