Introduction
Personal training-therapy, called personal therapy (Geller et al., Reference Geller, Norcross, Orlinsky, Geller, Norcross and Orlinsky2005), personal practice (J. Bennett-Levy and A. Finlay-Jones, unpublished observations), self-practice/self-reflection (SP/SR; e.g. Bennett-Levy, Reference Bennett-Levy2006) among other terms, has equivocal evidence (e.g. Macran and Shapiro, Reference Macran and Shapiro1998), and takes a range of formats (Geller et al., Reference Geller, Norcross, Orlinsky, Geller, Norcross and Orlinsky2005). It refers to a range of activities whose objective is the development of the therapist's ‘self’, professionally, personally or both, with the aim of improving therapeutic practice. The terms self-focused practice or training-therapy will be used interchangeably here, as we consider them to be more descriptive of goals of the activity. Associated research can be modality specific (e.g. Orlinsky et al., Reference Orlinsky, Schofield, Schroder and Kazantzis2011), qualitative (e.g. Daw and Joseph, Reference Daw and Joseph2007) and beset with methodological difficulties (Macran and Shapiro, Reference Macran and Shapiro1998). While most therapist training activities are rated, the impact of training-therapy is generally not. Available measures are often too long for use in everyday practice (e.g. the 392 item Development of Psychotherapists’ Common Core Questionnaire (DPCCQ); Orlinsky, Gerin, Davis et al., Reference Orlinsky, Gerin, Davis, Davis, Ambühl, Willutzki, Dazord, Davidson, Botermans, Aapro, Backx and Iahns1999). Though the Therapeutic Work Involvement Scales section of the DPCCQ has utility in regular reflection on personal growth (Orlinsky and Rønnestad, Reference Orlinsky and Rønnestad2005), it does not specifically target reflective practice. Likewise, the personal therapy section of the DPCCQ relates to the respondent's attitudes and experience of personal therapy (Orlinsky and Rønnestad, Reference Orlinsky and Rønnestad2005; Wiseman and Egozi, Reference Wiseman and Egozi2006), not the perceived impact of reflective practice. Surveys of influences on therapeutic practice exist (e.g. Lucock et al., Reference Lucock, Hall and Noble2006), while other measures are supervisor, not trainee, self-rated (e.g. Hayes et al., Reference Hayes, Gelso, Van Wagoner and Diemer1991).
Self-practice/self-reflection (SP/SR) has a growing, though mostly qualitative evidence base (Bennett-Levy et al., Reference Bennett-Levy, Turner, Beaty, Smith, Paterson and Farmer2001, Reference Bennett-Levy, Lee, Travers, Pohlman and Hamernick2003; Haarhoff et al., Reference Haarhoff, Gibson and Flett2011; Davis et al., Reference Davis, Thwaites, Freeston and Bennett-Levy2014), with few quantitative studies (e.g. Davis, Thwaites, Freeston and Bennett-Levy, Reference Bennett-Levy2008; Chaddock et al., Reference Chaddock, Thwaites, Bennett-Levy and Freeston2014). SP/SR may be a credible alternative to training-therapy in CBT (Chigwedere, Reference Chigwedere2015), and a measure of its impacts would increase opportunities for comparison with other approaches to training-therapy. This paper describes the exploratory factor analysis (EFA), reliability and validity testing of a measure of trainee therapists’ perceptions of the impact of their self-focused practice/training-therapy (e.g. SP/SR, personal therapy, group work).
Method
Development of the Self-focused Practice Questionnaire
Self-focused Practice Questionnaire (SfPQ) items were derived from six qualitative themes from a thematic analysis-based study into cognitive behavioural therapy (CBT) therapists’ experiences of SP/SR (Chigwedere, Reference Chigwedere2015). The themes were within two superordinate themes: (1) Personal-self (Awareness of current patters, Personal change, and Longitudinal formulation of ‘self’) and (2) Therapist-self (Declarative and procedural learning, Internalization of the model, and Development of empathy). They were used to derive a survey-style, initial set of items, which was reliability tested. Items were then modified to rate changes in participants’ self-perceived impact of self-focused practice. These steps are summarized as follows:
Stage 1: Generation and testing of the initial set of items
The initial set of items (N = 30) consisted of two subscales based on the qualitative superordinate themes and themes, rating (1) Therapist-self (n = 14 items) and (2) Personal-self (n = 16 questions) impacts of self-focused practice. These items were close in their wording, to qualitative study participants’ recorded statements. They were chosen because they were deemed by the researchers to be representative of the original qualitative themes. The items were initially constructed by the first author (C.C.) and then edited for comprehensibility following discussion with co-authors (G.D. and B.F.).
Item review
For further item review, a small convenience sample of ‘experts by participation in SP/SR’ (n = 6) completed the initial set of items and provided feedback. Qualitative item review involved interviews with three qualitative study participants (i.e. experts by participation) and an expert (i.e. SP/SR facilitator). Consequently, the items were reduced from an initial 52 to 30 items.
Validation of the initial set of items
From 26 potential participants recruited by email, a small non-randomized, convenience sample of 13 (age range 22–52 years) male (n = 5) and female (n = 8) volunteer psychological therapists from a local psychiatric hospital completed the questions. Responders (n = 13) were met face-to-face by C.C., for suitability assessment and consenting. To reduce the possibility of coercion, reasons for non-response were not sought from the 13 non-responders. Responders were deemed suitable if they met inclusion criteria, which were (1) participation in self-focused practice as part of their training or personal development process, and (2) current practice as a therapist.
Procedure for the validation of the initial set of items
For expediency, the questionnaires were hand delivered by C.C.; participants had approximately 1 week to respond, after which weekly email reminders were sent until the questionnaire was received, or for a maximum of 4 weeks. Retesting was between 2 and 4 weeks later, depending on participant availability. The set of items took 10–15 minutes to complete. Each item was rated on a 0 (not at all true of my experience) to 100% (completely true of my experience) scale.
The primary purpose of this stage was to (1) identify non-informative items (e.g. the ones on which most participants scored 0 or 100), (2) to test scale sensitivity and reliability, and (3) to confirm that the items were representative of the original qualitative study themes. The assumption was that participants would respond to representative items with a range of scores greater than 0, regardless of participation in the SP/SR qualitative study. The scale was duly changed from a percentage to a Likert-style scale, and items reworded to be sensitive to change in perception.
Stage 2: Refinement and validation of the final SfPQ
Participants : 112 participants from Trinity College Dublin (TCD) were recruited, made up of foundation or post-graduate diploma level CBT (n = 81), and first year counselling psychology doctoral trainees (n = 31). Counselling psychology participants were included because even though the questionnaire was developed from CBT-based SP/SR qualitative themes, it is a measure of perceived impact, not the quality or mode of self-focused practice per se.
Recruitment: Recruitment was through email, followed by face-to-face meetings. Inclusion criteria were participation in SP/SR or personal training-therapy, capacity to give informed consent, and membership of each course's first year cohort. Consenting trainees had up to 1 week to read the information sheets and return the consent forms. There were no penalties for non-consent, and consent could be withdrawn at any time.
Description of the final SfPQ
The SfPQ
The final SfPQ was a 49-item, two-scale measure of self-perceived impact of self-focused practice (regardless of mode of training-therapy). The first 22 items rate perceived Therapist-self impact, and the last 27 rate perceived Personal-self impact. Items are rated on a 4-point Likert scale: 0 (do not agree), 1 (slightly agree), 2 (definitely agree) and 3 (strongly agree).
Procedure: Questionnaires were administered at two time points between 3 and 4 weeks apart, in the first month of training, depending on participant accessibility.
Statistical analyses
Statistical analyses were performed using SPSS version 18 (http://www-01.ibm.com/software/analytics/spss/). Exploratory factor analysis (EFA) was preferred over principal components analysis (PCA) because EFA assumes the existence of latent variables, which give rise to the manifest variables (Osborne, Reference Osborne2014). Factors were chosen based on a combination of the Keiser criterion (Kaiser, Reference Kaiser1960, Reference Kaiser1970) – which proposes a lower bound Eigen factor of 1 for a meaningful factor – and examination of the scree plot (Cattell, Reference Cattell1966; Osbourne, Reference Osborne2014). Factor loadings were based on Stevens's (Reference Stevens1992) criteria, that for a sample of 100, a factor loading of .512 should be considered significant. This is a more conservative definition of significance than the usually accepted values of .3 or .4 (Field, Reference Field2005). Stevens (Reference Stevens1992) accepts the use of factor loadings of .3 or .4, but cautions that they should be reserved for higher sample sizes (300 and 200, respectively).
While the generally accepted range for Cronbach's alpha is >.7, cautious interpretation of guidelines and a lowering of the acceptable level in measurement of the type of data reported here have been recommended (Cortina, Reference Cortina1993; Kline, Reference Kline1999; Prince, Reference Prince, Prince, Stewart, Ford and Hotopf2003). Therefore, significance of Cronbach's alpha was set at >.60 (Nunnally and Bernstein, Reference Nunnally and Bernstein1994). Johnson and Gross's (Reference Johnson and Gross1997) intra-class correlation coefficient (ICC) reference values of ≤0.25 (small), 0.26–0.49 (low), 0.50–0.69 (moderate), 0.70–0.89 (high) and ≥0.90 (very high) were used. For Pearson's correlation coefficient (r), the values of r were considered significant if ≥.30, with a two-sided alpha value for significance p ≤ .01 or .05 as appropriate. Interpretation of the values of r have been suggested as ±.70 (strong, ±.50 (moderate), ±.30 (weak) and 0 (no relationship) (Ramsey, Reference Ramsey2011). Item ceiling and floor effects were acceptable at ≤20% of participants scoring at the lowest or highest score (Pakour et al., Reference Pakour, Nourozi, Molsted, Nourozi and Fridlund2011).
Results
Sample characteristics of the initial test cohort
The initial test cohort consisted of five males and eight females (χ2 = .077; p = .78), the majority of whom were from a psychiatric nursing background (Table 1). Most participants (n = 6) had received only SP/SR as a form of self-focused practice, followed by group-based approaches (n = 4).
Characteristics of the initial questionnaire validation participants
SfP: self-focused practice; CBT: cognitive behavioural therapy; SP/SR: self-practice and self-reflection
Stage 1: Content and construct validity of the initial questionnaire
Psychometric properties of the initial set of items
The initial items were representative of participants’ experiences of self-focused practice (Table 2), with good content and construct validity. The mean rating for each question (rated on a 0–100% agreement), ranged from 8.60 (SD 12.46) to 80.20 (SD 11.50) for the Therapist-self subscale, and 8.80 (SD 11.93) to 78.80 (SD 13.94) for Personal-self. For the combined subscales, the mean agreement was 52.70 (SD 24.78). Internal consistency reliability was good for the Personal-self subscale (α = .71) and moderate for Therapist-self (α = .60). Test–retest reliability was high for Personal-self (ICC = .81) and moderate for Therapist-self (ICC = .58).
Representativeness of the initial set of questions
Stage 2: Validation and reliability of the SfPQ
Sample characteristics for the SfPQ
Participant age ranged from 23 to 54 years (mean 33, SD 8.53). Females (n = 84) significantly outnumbered males (n = 28) (χ2 = 28.00; p = <.001), with more CBT (n = 81) than counselling (n = 31) trainees (χ2 = 87.88; p < .001). Mental health nurses (n = 49; 43.8%) formed the largest professional group, followed by counsellors (n = 40; 35.7%), with five (4.5%) psychotherapists, and three (2.7%) participants each from social work, alcohol counselling and occupational therapy. Physiotherapy, psychology, general practice and psychiatry had two (1.8%) participants each, with one (0.9%) activity therapist.
The Therapist-self subscale of the SfPQ
Floor and ceiling effects
Floor and ceiling effects on the 22 Therapist-self items, were within acceptable ranges (see Table 5).
Bivariate correlations
Items 2 (SP/SR/SfP/PT does not influence my practice) (r range .001–.22), 4 (SP/SR/SfP/PT is alien to me) (r range: .008–.18), 11 (Applying my specific approach to myself does not become easier with time) (r range: .01–.27), 14 (Applying my specific approach to myself is not at all complicated) (r range: .003–.24) and 18 (Right now, my understanding of the reasons for patients’ emotions and behaviours within therapy is better than it was) (r range: .006–.29) had weak correlations with all the other items. They were eliminated from any further analyses, leaving a 17-item subscale (see Table 3).
Therapist-self subscale factor loadings
Significant factors loadings are represented in bold type.
Exploratory factor analysis
The 17-item subscale was subjected to EFA, using oblique rotation (Promax with Kaiser normalization), and revealed five factors (Table 3). Factors 3, 4 and 5 had less than three variables with significant factor loadings (i.e. >.512, represented in bold type in Table 4) and were removed, leaving a 9-item, 2-factor subscale. This was supported by a review of the scree plot. Factors 1 and 2 were retained and labelled (1) Internalization of the Model (IoM), and (2) Development of Empathy (DoE).
Personal-self subscale factor loadings
The Personal-self subscale of the SfPQ
Floor and ceiling effects
Eighty-one per cent of respondents endorsed the highest response on items 19 (Right now, I believe that I am no more aware of my emotions than I ever was) and 20 (Right now, I believe that I am no more aware of my unhelpful ways of thinking than I ever was) and 83% on item 21 (Right now, I believe that I am no more aware of unhelpful ways in which I behave than I ever was). These items were eliminated.
Bivariate correlations
Bivariate correlations of the remaining 24 items showed that items 3 (Considering how I feel now, I feel less confident than I ever was) (r range: .008–.29) and 8 (My perception of myself now is that I might need to seek more therapy) (r range: .008–.25) had weak correlations with the other items. These items were excluded from any further analyses, leaving a 22-item questionnaire for the EFA.
EFA
Four factors were identified (Table 4). Factor 4 was a 2-item factor and was removed, leaving a 3-factor, 15-item subscale, confirmed by scree plot review. Items with significant factor loadings are represented in bold type in Table 4. The factors were labelled (1) Experience of Personal change (EoPC), (2) Felt-sense and Self-awareness (FSSA) and Awareness of Developmental Experiences (AoDE).
Internal consistency and test–retest reliability
Both subscales of the SfPQ and their factors had high internal consistency (α range: 81–90) (Table 5). Test–retest reliability was moderate to high for the Personal-self subscale and its factors (ICC range: 58–89) and high for Therapist-self and its factors (ICC range: 70–74) (Table 5).
Properties of the SfPQ and its components
α: Cronbach's alpha; ICC: intra-class correlation coefficient; IoM: internalization of the model; DoE: development of empathy; EoPC: experience of personal change; FSSA: felt-sense and self-awareness; AoDE: awareness of developmental experiences.
Intercorrelations of the SfPQ subscales and their factors
Both subscales were moderately intercorrelated (Table 6). Except for the AoDE factor, which moderately correlated with Personal-self, all factors had strong correlations with their respective subscale. Except for AoDE, which was not significantly correlated with the Therapist-self subscale (r = .17, p < .08), all other factors had moderate correlations with the alternative subscale. All factors had significant, low to moderate correlations with each other, except for AoDE and DoE (r = .04, p = .68) and AoDE and FSSA (r = .08, p = .40).
Correlations of the SfPQ scales and factors
r: Pearson's correlation coefficient (two-tailed); **correlation is significant at the ≤.001 (two-tailed) level; *correlation is significant at ≥.05 (two-tailed) level. TS: Therapist-self; PS: Personal-self; IM: internalization of the model; DE: development of empathy; EPC: experience of personal change; FSSA: felt-sense and self-awareness; AoDE: awareness of developmental experiences.
Discussion
The SfPQ is a measure of the self-perceived impact of self-focused practice/training-therapy, in its various formats (e.g. SP/SR, and therapist assisted personal training-therapy). The SfPQ's reliability and validity and EFA were examined in a convenience sample of 112 trainees.
Both SfPQ subscales and their factors had acceptable internal consistency reliability, comparable to widely utilized measures, such as the Empathy Scale (Burns and Nolen-Koeksema, Reference Burns and Noelen-Hoeskema1992), the Working Alliance Inventory (WAI; Hovarth and Greenberg, Reference Horvath and Greenberg1989) and the DPCCQ (Orlinsky and Rønnestad, Reference Orlinsky and Rønnestad2005). The moderate to high intercorrelations between subscales and their factors supported the two-scale structure. Only the Awareness of Developmental Experiences factor was moderately (not strongly) correlated with its relevant subscale. The significant low to moderate intercorrelations amongst the factors are acceptable for a measure of psychological concepts (Kline, Reference Kline1999). However, non-significant correlations between Awareness of Developmental Experiences and the Therapist-self scale, Experience of Personal Change and Felt-sense/Self-awareness make intuitive sense. Awareness of Developmental Experiences describes self-referential, retrospective awareness, which may not directly impact on the concept of self-as-therapist, understanding of the model, or one's ‘felt-sense’. The moderate intercorrelation between the two subscales supports the EFA, which supports existing self- and therapist-schema theory (e.g. Bennett-Levy, Reference Bennett-Levy2006; Geller et al., Reference Geller, Norcross, Orlinsky, Geller, Norcross and Orlinsky2005).
Though the original qualitative study (Chigwedere, Reference Chigwedere2015) identified three Therapist-self themes, the EFA only revealed two (Internalization of the Model and Development of Empathy), which excluded the qualitative theme of Declarative and Procedural Knowledge. This makes both intuitive and theoretical sense. Model internalization may encompass knowledge development, subsumed within the DPR model's conceptual/technical skills (Bennett-Levy, Reference Bennett-Levy2006; Bennett-Levy, Thwaites, Chaddock, and Davis, 2009). However, Development of Empathy may be representative of interpersonal perceptual skills (Bennett-Levy, Reference Bennett-Levy2006; Bennett-Levy et al., 2009). Thus the Therapist-self subscale acknowledges both the self-referential and other-directed aspects of the therapist. The Personal-self subscale also tallies with existing theory. Therapist self-care aspects of self-focused practice are widely acknowledged (e.g. Bennett-Levy et al., Reference Bennett-Levy, Thwaites, Haarhoff and Perry2015). Beck (Reference Beck, Beck, Rush, Shaw and Emery1979) highlighted the importance of change in therapists’ therapy, while self-awareness (Rønnestad and Skovholt, Reference Rønnestad and Skovholt2013) is important in literature.
Study limitations include the sample's size and its diversity. However, the SfPQ is a measure of ‘perceived impact’ of self-focused practice, not the practice or its quality, explaining the sample's diversity. Due to resource limitations and difficulty identifying a similar measure, concurrent validity and objective measures were not examined. Psychodynamic and other therapists may criticize the subscales for not explicitly identifying issues of transference and counter transference. However, the questions are stated in a way that allows respondents to answer according to their own theoretical orientation. For example, questions relating to issues ‘developed in my childhood’, ‘effects of beliefs from my past experiences’ and ‘reasons for patients’ emotions and behaviours in therapy’ could be answered with transference/countertransference in mind.
Finally, throughout their careers, therapists seek therapy for a broad range of reasons not only for professional development (Norcross and Connor, Reference Norcross, Connor, Geller, Norcross and Orlinsky2005; Daw and Joseph, Reference Daw and Joseph2007). The SfPQ is a tool for rating self-focused practice, in its various guises, for the purposes of professional development. It is not a measure of therapeutic impact, which might be appropriate when therapy is primarily sought to alleviate distress. In conclusion, these preliminary results support the SfPQ's acceptability by a range of therapist levels (from foundation to post-graduate diploma level in CBT, to doctoral counselling psychology) and its validity and reliability. Though further examination may be needed, these are encouraging results. The SfPQ should be considered for incorporation into personal training-therapy rating practices of training and accreditation institutions that utilise personal training therapy. To our knowledge, a similar measure does not exist. Use of the SfPQ or similar measure may remedy the current situation, where many training institutions do not evaluate training-therapy.
Acknowledgements
We would like to thank Professor James Bennett-Levy, Dr Richard Thwaites, Dr Ladislav Timulak, Professor Mark Freeston and Dr Michael Duffy, as well as our participants.
Conflicts of interest: None of the authors report any conflict of interest.
Ethical standards: Craig Chigwedere, Brian Fitzmaurice and Gary Donohoe assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, and its most recent revision.
Comments
No Comments have been published for this article.