Many models of psychopathology and most theories of psychotherapy assume that how a person experiences or interprets events is a critical factor in determining emotional maladjustment and psychopathology. The early pioneers in cognitive behavioral therapy, Albert Ellis and Aaron Beck, were trained in psychoanalytic theory. In their subsequent theorizing, they maintained Freud's view that irrational beliefs play a central role in creating and maintaining psychopathology.
Theories of Maladaptive Beliefs
There are varying underlying assumptions among different theorists and models about maladaptive beliefs. For example, psychoanalytic theorists (e.g., Bieber, Reference Bieber1980; Luyten & Blatt, Reference Luyten and Blatt2013; Wallerstein, Reference Wallerstein2000; Weiss, Reference Weiss1998) suggest that thoughts and beliefs about past attachment experiences are important factors in the development of unconscious anxiety, which in turn leads to psychopathology. In contrast, cognitive therapy models of anxiety and depression propose that irrational beliefs (Ellis, Reference Ellis2010), dysfunctional core beliefs (Beck, Reference Beck1967; Beck, Rush, Shaw, & Emery, Reference Beck, Rush, Shaw and Emery1979), or maladaptive schemas (Beck, Reference Beck2011) are primary risk factors for clinical disorders. According to this cognitive therapy model, psychopathologies stem from thinking errors, such as all or nothing thinking, and maladaptive thought processes, such as attributions of personal responsibility for adverse events and negative beliefs about the futureFootnote 1.
Moreover, control-mastery theory, an integrated cognitive-psychodynamic-relational theory of how psychopathology develops and how psychotherapy works (Silberschatz, Reference Silberschatz2005; Weiss, Reference Weiss1993), posits that early adverse experiences are internalized as conscious or unconscious pathogenic beliefs that are specific to the individual. Consider, for example, a young man who was an only child that was raised by a very isolated, overwhelmed single parent. While growing up he noticed that his parent became agitated and depressed when he stayed away from home to participate in sports and other after school activities. Based on these experiences he developed the belief that he was responsible for others’ feelings, contributing to his severe anxiety in close relationships. Based on the same experiences, another person might develop beliefs that the world is dangerous, leading to anxiety in public spaces and panic attacks. Weiss referred to these as “pathogenic beliefs” because they are painful convictions about self and others that cause severe emotional distress and psychopathology.
Despite differences in emphasis, the importance of maladaptive beliefs is emphasized in most therapy models including internal working models (Bowlby, Reference Bowlby1988), negative cognitive schemata (Young, Reference Young1990), cognitive maps (Tolman, Reference Tolman1948), family myths (Ferreira, Reference Ferreira1966), interpersonal expectations (Beebe & Lachmann, Reference Beebe and Lachmann1998), attachment representations (Main, Kaplan, & Cassidy, Reference Main, Kaplan and Cassidy1985), and cyclical psychodynamics (Wachtel, Reference Wachtel1994). These different conceptualizations are similar in that they address a repetitive pattern of emotion-laden beliefs that developed in younger years for adaptive reasons and continue to influence current internal and external experiences of the world.
Existing Belief Measures
These belief patterns have been categorized in different ways. The rational emotive therapy model developed by Ellis (Reference Ellis2010), for example, points to four primary categories of irrational beliefs that are theoretically derived including demandingness (i.e., absolute/inflexible requirements), awfulizing (i.e., catastrophizing), frustration intolerance, and negative global evaluation. Based on theory, Young (Reference Young1990) categorized schemata in five domains (e.g., categories of disconnection and rejection, impaired autonomy and performance, impaired limits, other directedness, and overvigilance and inhibition; see Abela & Hankin, Reference Abela, Hankin, Nolen-Hoeksema and Hilt2009). Beck's cognitive triad distinguishes three main types of core beliefs, those about the self, others, and the world.
More than 25 different irrational belief questionnaires are currently available based on the models proposed by Ellis (Reference Ellis2010) and Beck, (Reference Beck1967, Reference Beck2011). For a review of irrational belief measures that are based on the rational emotive therapy model see Terjesen, Salhany, and Sciutto, (Reference Terjesen, Salhany and Sciutto2009), and see Bridges and Harnesh (Reference Bridges and Harnish2010) for a review of measures that are based on both models. Although the large number of these measures illustrates the theoretical relevance of irrational beliefs, recent reviews suggest that these existing belief measures suffer from psychometric deficiencies and conceptual problems (Bridges & Harnish, Reference Bridges and Harnish2010; Terjesen et al., Reference Terjesen, Salhany and Sciutto2009). Moreover, the existing belief measures have been developed in a “top down” model in the sense that the items are based entirely on theoretically proposed diagnostic or symptom-based categories such as demandingness, awfulizing, all or nothing thinking, or negative attributions. For example, the Dysfunctional Attitudes Scale (see D'Alessandro & Burton, Reference D'Alessandro and Burton2006; Lau, Segal, & Williams, Reference Lau, Segal and Williams2004) was developed to assess depressogenic schemas in people that had been diagnosed with depression. The Dysfunctional Attitudes Scale includes items, such as “My value as a person depends greatly on what others think of me” and “If others dislike you, you cannot be happy.” Another example, The Belief Scale (Malouff & Schutte, Reference Malouff and Schutte1986), was specifically designed to assess the beliefs that are related to a symptom-based category (e.g., demandingness). It includes items such as “To be a worthwhile person, I must be thoroughly competent in everything I do” and “To be happy, I must maintain the approval of all the persons I consider significant.”
While such measures (and the belief categories from which they are derived) may be highly relevant to the psychopathology for which they were designed, they are likely to be irrelevant for patients who present with other types of symptoms. In order to broaden the scope of the belief measure, the PBS was intended to reflect common psychopathology more generally.
Furthermore, even people with the same diagnosis and symptomatology are heterogeneous in their history and belief patterns (Lingiardi & McWilliams, Reference Lingiardi and McWilliams2015; Persons, Reference Persons1986; Persons & Hong, Reference Persons, Hong, Tarrier and Johnson2016; Silberschatz, Reference Silberschatz2017) and people with different diagnoses often show similar belief patterns (Kush, Reference Kush2004). Arguably, nomothetic measures like the ones described do not always seem to capture beliefs that are unrelated to the particular diagnosis. Individualized belief assessment might be relevant for capturing data that describes the issues that are of the greatest concern to patients (Sales, Neves, Alves, & Ashworth, Reference Sales, Neves, Alves and Ashworth2017).
Pathogenic Beliefs
In contrast to approaches that emphasize proscribed categories of beliefs based on theory or diagnoses, patient-centered, individualized treatment models emphasize pathogenic beliefs as being case-specific (idiographic) and arising from particular experiences.
There is a considerable body of research showing that such idiographic pathogenic beliefs can be reliably identified by trained clinical judges (e.g., Curtis, Silberschatz, Sampson, & Weiss, Reference Curtis, Silberschatz, Sampson and Weiss1994; Rosenberg, Silberschatz, Curtis, Sampson, & Weiss, Reference Rosenberg, Silberschatz, Curtis, Sampson and Weiss1986; for a review of this research, see Curtis & Silberschatz, Reference Curtis, Silberschatz and Silberschatz2005) and that therapeutic progress is significantly correlated with therapists’ successfully disconfirming their patients’ pathogenic beliefs (e.g., Silberschatz & Curtis, Reference Silberschatz and Curtis1993; for an overview see Silberschatz, Reference Silberschatz2005, chapter 11). Research also suggests that idiographically derived pathogenic beliefs can serve as a basis for a transdiagnostic nomothetic measure of internalized beliefs that addresses cognitions that are related to common psychopathology symptoms that are present in people with a wide range of diagnoses (Sammet et al., Reference Sammet, Leichsenring, Schauenburg and Andreas2007; Silberschatz & Aafjes-van Doorn, Reference Silberschatz and Aafjes-van Doorn2017). This transdiagnostic perspective of psychopathology is in line with emerging conceptualizations of psychopathology such as the National Institute of Mental Health's Research Domain Criteria Initiative (Insel et al., Reference Insel, Cuthbert, Garvey, Heinssen, Pine, Quinn and Wang2010) and the newly developed Psychodynamic Diagnostic Manual—2 (Lingiardi & McWilliams, Reference Lingiardi and McWilliams2015).
The Pathogenic Belief Scale (PBS)
The PBS (Silberschatz, Reference Silberschatz2008; Silberschatz & Aafjes-van Doorn, Reference Silberschatz and Aafjes-van Doorn2017) was developed by using a “bottom up” approach (in contrast to the “top down” measures referenced above), which means that the items were based on what individuals report rather than on what a particular theorist proposes. Such an approach makes no a priori assumptions about categories of beliefs, focusing instead on a compilation of beliefs that is expressed by patients. The items in the PBS were drawn from psychotherapy cases rather than from particular theoretical models. Therefore, the PBS can be seen as patient-centered. It was not designed to test theoretical models or identify diagnostic categories such as depression or anxiety (as was the case in the irrational belief measures that were reviewed by Bridges and Harnish, Reference Bridges and Harnish2010). Instead, the PBS was designed to measure transdiagnostic internalized beliefs that were clinically derived and that address common psychopathology symptoms more generally.
Pathogenic Belief Scale Construction
Items for the PBS were derived from idiographic case formulations based on the intake sessions of 21 psychotherapy cases, which included both time-limited cognitive-behavioral and psychodynamic therapies as well as long-term psychodynamic treatments. The psychiatric presentations of these cases were representative of the types of problems that are frequently seen in outpatient psychotherapy. However, due to the lack of formal diagnostic assessments for these cases, it is unclear whether they covered the full spectrum of psychopathology.
Verbatim transcripts of the intake sessions from these cases were read by experienced psychotherapists (three to six clinicians). Following training in the control-mastery case formulation method (e.g., Curtis et al., Reference Curtis, Silberschatz, Sampson and Weiss1994; Rosenberg et al., Reference Rosenberg, Silberschatz, Curtis, Sampson and Weiss1986), they independently prepared lists of pathogenic beliefs for each case with high inter-rater reliability (ICC = .80–.96; Silberschatz & Aafjes-van Doorn, Reference Silberschatz and Aafjes-van Doorn2017). A total of 144 (nonredundant) pathogenic beliefs were generated. A team of four judges (two experienced, licensed clinicians and two advanced doctoral-level clinical psychology interns) independently categorized the idiographic beliefs into nomothetic categories, resulting in the 59-item inventory. Examples of idiographic beliefs and their corresponding nomothetic category include the following: “Joe worried that having his own independent life would hurt his parents” and “Jane believed that if she were to pursue her own interests and goals she would feel that she is abandoning her family” are contained in the nomothetic category, “Separating from parents or loved ones would be hurtful, disloyal, or make them feel abandoned.” “Mary feels that she is an academic failure and has bitterly disappointed her family” is represented by the nomothetic category, “S/he did not live up to his/her family's high expectations.” “Larry believes that if he is successful in his career his older brother will feel inferior and jealous” is nomothetically categorized as “S/he should play down achievements or success in order to avoid diminishing, offending, or emasculating others.”
The initial validation findings of the PBS (Silberschatz & Aafjes-van Doorn, Reference Silberschatz and Aafjes-van Doorn2017) indicated that the PBS-59 shared some variance with existing measures of perceived negative parenting style and current adult psychopathology symptoms. Thirty-one percent of variance in the occurrence of adult psychopathology symptoms in a diverse community-based sample was explained by pathogenic beliefs, and negative parenting style did not add significantly to the prediction of common psychopathology symptoms. Pathogenic beliefs on the PBS played a mediating role in the relationship between reported negative parenting style and pathology. Based on these findings, the authors suggested that the predictive power of the reported general negative parenting style depends on the extent to which one develops pathogenic beliefs, rather than on the negative parenting style itself.
Our objective in the present study was to operationalize a transtheoretical understanding of pathogenic beliefs that reflects experiences that are identified by patients themselves (bottom-up), with the goal of remaining unbiased by specific therapy models and theories. Following up on the previous development and initial validation of the PBS (Silberschatz & Aafjes-van Doorn, Reference Silberschatz and Aafjes-van Doorn2017), the current work reports on an exploratory factor analysis (EFA) of the measure. More specifically, we assessed the psychometric properties, construct validity, and convergent validity of the PBS in a heterogeneous sample that included both community participants and clinic participants.
This study builds on the previous validation study by adding a large clinical population and identifying underlying factors, which is not only important for future etiological research but also yields clinically meaningful results (Watson, Clark, & Harkness, Reference Watson, Clark and Harkness1994). Grouping items together to get a sense of each factor is more valid and informative than considering each item individually, which means that subscales allow for more reliable generalizations than individual items do, reflecting underlying constructs that can be considered in the formulations of patients.
Method
Participants
The sample consisted of two heterogeneous groups of participants, a clinical group of 246 adults who sought psychotherapy in an outpatient clinic and a community group of 732 adults who were recruited online. We wanted to recruit a diverse sample of people representing various ages, education levels, and socioeconomic backgrounds. The data from the clinic participants were collected as part of the standard clinic procedures. The community sample was recruited via word of mouth, service organizations, and online advertisement on websites where volunteers could participate, including student websites (e.g., http://psych.hanover.edu/research/exponnet.html and http://www.socialpsychology.org/expts.htm#pinterpersonal), Craigslist, and psychology listservs (e.g., psychotherapyresearch.org) with a link that directed them to surveymonkey.com. Detailed demographic information for the overall sample is presented in Table 1. The sample of clinic participants was not restricted by psychiatric diagnosis or level of functioning. In this outpatient clinic, no DSM diagnostic assessments were conducted, so no formal DSM diagnoses are available for the sample of clinic participants. As part of the clinic protocol, all of the patients completed the Beck Anxiety Inventory (BAI; Beck & Steer, Reference Beck and Steer1993) and the Beck Depression Inventory—II (BDI-II; Beck, Steer, & Brown, Reference Beck, Steer and Brown1996). In total, 126 participants were above the clinical cutoff on either measure (51.22%). Based on the clinical impressions of the clinic supervisors and as also reflected in the BAI and BDI scores, the majority of the outpatients suffered from symptoms of anxiety and/or depression.
Descriptive statistics of the clinical sample, community sample, and the total sample
Note: *The BAI, BDI, and OQ-45 were available only for the clinical subsample (n = 246 patients). **Indicates that the score is above the clinical cutoff (BDI > 17; OQ-45 > 63). MOPS = Measure of Parental Style (15 items); PBS = Pathogenic Beliefs Scale (59- and 34-item versions); BAI = Beck Anxiety Inventory (21 items); BDI = Beck Depression Inventory—II (21 items); OQ-45: Outcome Questionnaire-45 (45 items); Range on the MOPS is 0–3; Range on the PBS is 0–4; Range on the BAI is 0–3; Range on the BDI is 0–3; Range on the OQ-45 is 0–4.
Measures
The Pathogenic Beliefs Scale (PBS)
The PBS is a 59-item inventory of self-reported pathogenic beliefs that were derived from previous psychotherapy research. Participants were asked to read each of the items and rate on a 5-point Likert-type scale the degree to which it was applicable or accurate (0 = Not at all, 1 = Slightly, 2 = Moderately, 3 = Highly, 4 = Completely). Examples of representative items on the PBS include “I need to defer to others instead of pursuing my own ideas, needs, or interests”; “It is wrong, threatening, or disloyal to surpass one's family or significant others”; “I do not deserve to be happy”; and “I believe that I must be perfect in order to feel good about myself.” Higher scores reflect the identification of higher levels of pathogenic beliefs, with mean scores ranging from 0–4 and total scores ranging from 0–157. In previous research, the PBS scores were shown to be reliable (α = .96) and valid (Silberschatz & Aafjes-van Doorn, Reference Silberschatz and Aafjes-van Doorn2017), and the PBS has recently been translated in Thai and validated in patients with depressive disorder in Thailand (Neelapaijit, Wongpakaran, Wongpakaran, & Thongpibul, Reference Neelapaijit, Wongpakaran, Wongpakaran and Thongpibul2017; Neelapaijit, Wongpakaran, Wongpakaran, Thongpibul, & Wedding, Reference Neelapaijit, Wongpakaran, Wongpakaran, Thongpibul and Wedding2018). In the current sample, the internal consistency of the PBS was similar (α = .97).
Measure of Parental Style (MOPS)
The MOPS (Parker et al., Reference Parker, Roussos, Hadzi-Pavlovic, Mitchell, Wilhelm and Austin1997) is a 15-item measure of perceived parenting characteristics that contribute to the quality of the parent–child bond that evolved from the earlier 25-item Parental Bonding Instrument (PBI; Parker, Tupling, & Brown, Reference Parker, Roussos, Hadzi-Pavlovic, Mitchell, Wilhelm and Austin1997; Parker et al., Reference Parker, Roussos, Hadzi-Pavlovic, Mitchell, Wilhelm and Austin1997). The PBI probes adult recollections of parental behaviors and attitudes during the subject's childhood and has been used as a parenting style measure in many studies on the parenting–pathology relationship (cognitive vulnerability and parenting; Alloy, Abramson, Smith, Gibb, & Neeren, Reference Alloy, Abramson, Smith, Gibb and Neeren2006). Two decades later, Parker et al. (Reference Parker, Roussos, Hadzi-Pavlovic, Mitchell, Wilhelm and Austin1997) designed a modified version of the PBI, named the Measure of Parental Style (MOPS). The MOPS can be used as a broad-brush measure of the likelihood of exposure to dysfunctional parenting, allowing the level of any abuse to be simply quantified. The MOPS measures three subscales: parental abuse, indifference, and overcontrol. It consists of a total of 15 statements (e.g., “verbally abusive,” “ignored me,” “overprotective of me”) that are scored on a 4-point Likert-type scale, and participants recall about either their relationship with their mother (maternal form) or father (paternal form) during their first 16 years of life. In this study, the MOPS was completed with reference to both mother and father and the scores were averaged (Parker et al., Reference Parker, Roussos, Hadzi-Pavlovic, Mitchell, Wilhelm and Austin1997), as was done in several other studies (e.g., Dalgleish et al., Reference Dalgleish, Tchanturia, Serpell, Hems, Yiend, de Silva and Treasure2003; Valiente, Romero, Hervas, & Espinosa., Reference Valiente, Romero, Hervas and Espinosa2014 for the PBI), resulting in a possible range from a minimum of 0 to a maximum score of 4. Due to the nonsignificant, very minimal differences in reported scores on the maternal and paternal MOPS forms in our samples (see Table 1), a combined total mean MOPS score (i.e., general parental score) was used in further analyses. The level of disrupted parental practices is reflected in the total summed MOPS score, which ranges from 0 to 63, with a higher score indicating more reported general adverse parenting experiences. The MOPS has recently been validated in nonclinical studies (Picardi et al., Reference Picardi, Tarsitani, Toni, Maraone, Roselli, Fabi and Biondi2013) as well as in clinical samples, relating adverse parenting with psychopathology (e.g., Fletcher, Parker, Bayes, Paterson, & McClure, Reference Fletcher, Parker, Bayes, Paterson and McClure2014; Parvez, Reference Parvez2013; Vracotas & Malla, Reference Vracotas and Malla2011). In the current sample the internal reliability was α = .89.
Beck Anxiety Inventory (BAI)
The BAI (Beck & Steer, Reference Beck and Steer1993) is a measure of anxiety symptoms that is often used in psychiatric outpatient services. The BAI consists of 21 questions regarding how the individual has been feeling in terms of common symptoms of anxiety in the past week. The BAI is designed to distinguish anxiety symptoms from depressive symptoms based on symptoms recently experienced including numbness, hot and cold sweats, or feelings of dread. The BAI demonstrates high internal reliability (α = .94) and good factorial and discriminant validity (Creamer, Foran, & Bell,Reference Creamer, Foran and Bell1995; Fydrich, Dowdall, & Chambless, Reference Fydrich, Dowdall and Chambless1992; Steer, Ranieri, Beck, & Clark, Reference Steer, Ranieri, Beck and Clark1993) in outpatient and nonclinical samples. The total scores range from 0 to 63 and the clinical cutoff score is 16 (Beck & Steer, Reference Beck and Steer1993. In the current sample the internal reliability was α = .92.
Beck Depression Inventory—II (BDI-II)
The BDI-II (Beck, Steer, & Brown, Reference Beck, Steer and Brown1996) is a 21-item self-report measure that assesses common depressive symptoms, such as depressed mood, hopelessness, suicidal ideation, sleep disturbance, and appetite change. The BDI-II is commonly used in psychiatric outpatient services and has very good internal consistency (split-half Pearson = .93). It also has well-established content validity and is good at differentiating between depressed and nondepressed individuals (Beck et al., Reference Beck, Steer and Brown1996; Richter, Werner, Heerlein, Kraus, & Sauer, Reference Richter, Werner, Heerlein, Kraus and Sauer1998). The total scores range from 0 to 45 and the clinical cutoff score is 17 (Beck, Steer, & Brown, Reference Beck, Steer and Brown1996). In the current sample the internal reliability was α = .90.
Outcome Questionnaire-45.2 (OQ-45)
The OQ-45 (Lambert et al., Reference Lambert, Burlingame, Umphress, Hansen, Vermeersch, Clouse and Yanchar1996) is a commonly used general measure of self-reported symptoms of distress (mainly anxiety, depression, somatic problems, and stress) as well as difficulties in interpersonal relationships, social role (such as work or school), and the general quality of life. The OQ-45 consists of 45 items on a 5-point Likert-type (never to almost always) scale. The total score is calculated by summing all 45 items. The higher the score, the more disturbed the patient. Scores range from 0 to180 and the clinical cutoff score is 63 (which indicates symptoms of clinical significance). In the current sample the internal reliability was α = .90.
Procedure
Recruitment
A cross-sectional survey design was used. All of the participants volunteered to take part in the study without any monetary reward and were given identical packets with the measures presented in the same order. Informed consent was obtained from all of the participants that were included in the study. All of the community participants that were recruited online were asked to read and sign the informed consent statement before starting the online survey, and they completed the PBS and the MOPS. Clinic participants were asked for their consent at intake and received a standard battery of paper and pencil assessment measures, including the PBS and MOPS and measures of psychopathology (BAI, BDI, OQ-45). All of the participants were informed that their participation was voluntary and that they would not be compensated. Strict confidentiality was assured by the fact that the survey was anonymous and that no identifying information was requested.
Data analyses
We thoroughly cleaned the data before analysis, and we screened the data for random responding on the outcome measures by checking for duplicate responses, skewed results (responses that have all the same answers or have certain patterns), missing data, and data inconsistencies. Besides looking at the raw data, we used graphical methods (scatter, box plots, and histograms) for detecting outliers. No participant was removed due to a suspicious responding pattern. All of the statistical analyses were performed with R 3.3.2 (R Core Team, 2016) using the psych (Revelle, Reference Revelle2016) packages.
Exploratory factor analysis (EFA) was used to identify latent constructs and to arrive at a parsimonious representation of the associations among the measured variables (rather than principal component analysis, which is used when the goal is data-reduction). We conducted the EFA on the total sample of 978 participants, which is more than adequate for conducting such analyses (Bentler, Reference Bentler2007; Fabrigar, Wegener, MacCallum, & Strahan, Reference Fabrigar, Wegener, MacCallum and Strahan1999; Kelloway, Reference Kelloway2015). Convergent validity analyses were conducted for the 246 clinic participants because the psychopathology measures were available only for that sample (BAI, BDI, OQ-45). Given that there was insufficient basis to specify an a priori model, no a priori number of common factors were specified and no restrictions were placed on the patterns of relations between the common factors and the measured variables (i.e., the factor loadings). In line with the development of the PBS, the EFA, as a data-driven approach, provides procedures for determining an appropriate number of factors and pattern of factor loadings primarily from the data.
Results
Preliminary Analyses
Prior to the main analyses, the collected data were examined for data entry errors, completeness, and missing values. Missing data did not appear to pose any threat to statistical validity (Cohen, Cohen, West, & Aiken, Reference Cohen, Cohen, West and Aiken1983). Approximately 5% of the responses for any variable reflected missing data and no patterns in missing values could be identified.
Exploratory Factor Analysis
The 59 items of the PBS were analyzed by using EFA, with maximum likelihood as the factor extraction method and promax rotationFootnote 2 (Fabrigar et al., Reference Fabrigar, Wegener, MacCallum and Strahan1999). The scree plot, parallel analysis, eigenvalues, and very simple structure (Revelle & Rocklin, Reference Revelle and Rocklin1979) were used to determine the optimal number of factors to extract. The very simple structure for the PBS-59 indicated that one or two factors were optimal to extract, while a parallel analysis suggested that four factors were optimal. The values of the first six eigenvalues in the unrotated factor solution were 23.69, 2.69, 1.91, 1.66, 1.47, and 1.34. After considering these results and examining the scree plot (see Figure 1), we extracted three factors. The three-factor solution of the 59-item PBS accounted for 45% of the variance.
Scree plot of the 59-item Pathogenic Beliefs Scale.
Subsequently, 17 of the 59 items were removed because they saturated too weakly (<0.40)Footnote 3 with the dimension to which they belonged. Eight additional items were removed because they loaded on multiple factors, which resulted in a 34-item scale. Based on the widely accepted extraction rules of Kaiser (Reference Kaiser1960) and Cattell (Reference Cattell1965) for factor analysis results, we identified a three-factor structure for the PBS-34 (three factors with an eigenvalue > 1 as well as three factors above the break point in the scree plot, see Figure 2). The resulting three-factor structure for the PBS-34 explained 51% of the variance in the 34-item scale.
Scree plot of the 34-item Pathogenic Beliefs Scale.
The appropriateness of the three-factor solution of the PBS-34 was also evaluated through the degrees-of-freedom-corrected root mean square residual (RMSR), which measures the average discrepancy between the observed and reproduced correlation matrices. With three factors the RMSR was 0.04, well within the recommended threshold of 0.08 (Hu & Bentler, Reference Hu and Bentler1999). Additional fit indices were examined, and the comparative fit index = 0.91; the Tucker–Lewis index = 0.90; and the Root Mean Square Error of Approximation = 0.061, 90% CI [0.058, 0.063], suggesting that three factors were sufficient to reproduce the correlation matrix among the 34 items of the PBS. See Table 2 for the fit indices of the different factor solutions of the PBS-59 and PBS-34.
Fit indices for different numbers of factors extracted from the PBS-59 and PBS-34
Note: RMSR = Root Mean Square Residual; TLI = Tucker–Lewis Index; CFI = Comparative Fit Index; BIC = Bayesian Information Criterion; RMSEA = Root Mean Square Error of Approximation. N = 978.
All of the items saturated adequately in the expected dimensions, revealing a simple structure with factor loadings ranging from 0.84 to 0.42 for Factor 1 (labeled “cannot rely on others”), from 0.98 to 0.41 for Factor 2 (labeled “undeserving”), and from 0.78 to 0.40 for Factor 3 (labeled “interpersonal guilt”). See Table 3 for the factor loadings of all 34 items. Factor 1 accounted for 20% of the variance, Factor 2 accounted for 14%, and Factor 3 accounted for 13%. The factor, cannot rely on others, included items such as “S/he cannot rely on others to maintain a stable, strong attachment” and “Others will be emotionally unreliable or rejecting.” The undeserving factor included items such as “S/he does not deserve to be happy” and “S/he is unworthy and deserves very little in life.” The interpersonal guilt factor included the items, “It is wrong, threatening, or disloyal to surpass one's family or significant others” and “S/he must remain excessively involved with parents or loved ones.” The original 59-item PBS with the final 34-item PBS is included in the Appendix.
Factor loadings for the Pathogenic Belief Scale items
Note: PBS = Pathogenic Belief Scale; Total PBS items = 34. Two items did not load greater than 0.40 on the factor, cannot rely on others, and they are not displayed—item 52 (loading = 0.39) and item 30 (loading = 0.38). The PBS items that are shown in bold have a factor loading ≥ 0.40.
Reliability
The PBS-34 total scale was found to have a high internal reliability coefficient (Cronbach alpha = 0.95). The internal reliability coefficients of the three subscales were .92, .84, and .84.
Convergent Validity
The assessment of convergent validity with the BAI, BDI, OQ-45, and MOPS was based on the theoretical correspondence of the constructs of psychopathology (anxiety, depression, and general distress) and adverse childhood experiences. The PBS-34 total score was found to have significant medium to large correlations (Cohen, Reference Cohen1992) with commonly used measures of psychopathology and life functioning including the BAI (r = .40), BDI (r = .55), OQ45.2 (r = .59), and the MOPS (r = .37). The three individual subscales of the PBS were also found to have medium to large correlations with these measures (r = .22–.63), indicating that these constructs were highly convergent with the PBS-34 (see Table 4).
Pearson product-moment correlations between the PBS-34, BAI, BDI, OQ-45, and MOPS in the total sample (N = 978)
Note: aThe BAI, BDI, and OQ were available only for the clinical subsample (N = 206, 210, and 176, respectively). PBS-34 = Pathogenic Beliefs Scale Total Scores; BAI = Beck Anxiety Inventory; BDI = Beck Depression Inventory—II; OQ: Outcome Questionnaire-45; MOPS = Measure of Parental Style. *p < .05 **p < .01.
We compared the 95% confidence intervals for each correlation in order to determine whether the values significantly differed for the different PBS subscales. The OQ-45 was more highly related to the cannot rely on others and undeserving subscales than to the subscale for interpersonal guilt. The correlation between the OQ-45 and cannot rely on others, r = .63, 95% CI = [.52, .71], differed significantly from the correlation between the OQ-45 and interpersonal guilt, r = .31, 95% CI = [.15, .44], but not from the correlation between OQ-45 and undeserving, r = .53, 95% CI = [.41, .63]. The correlation between the OQ-45 and undeserving was not significantly different from the correlation between OQ-45 and interpersonal guilt. The BDI was most highly associated with the subscales cannot rely on others and undeserving and only showed a relatively small association with interpersonal guilt. However, the confidence intervals indicated that the correlations between the BDI and cannot rely on others, r = .53, 95% CI = [.42, .62], undeserving, r = .56, 95% CI = [.46, .65], and interpersonal guilt, r = .38, 95% CI = [.24, .49], did not differ significantly from each other. Similarly, the BAI was more highly associated with cannot rely on others (r =.40) than with the other two subscales (r =.27 and .29, respectively), but this difference was not significant.
Compared with the psychopathology measures, the MOPS showed lower correlations with the PBS subscales (r = .22–.40). The correlation between the MOPS and cannot rely on others, r = .40, 95% CI = [.27, .51], did not differ significantly from that between the MOPS and undeserving, r = .23, 95% CI = [.10, .36], or from that between the MOPS and interpersonal guilt, r = .22, 95% CI = [.08, .35].
Both the community and clinical samples completed the PBS and the MOPS. Across the two samples, there were no differences in the correlations between the MOPS and cannot rely on others and undeserving. However, there was a significant difference in the correlation between the MOPS and interpersonal guilt in the two samples. In the community sample, this correlation was not significant, r = –.01, p > .05, whereas in the clinical sample this correlation was significant, r = .22, 95% CI = [.08, .35].
Discussion
The primary purpose of the current study was to develop a clinically derived (based on idiographic data) self-report measure of pathogenic belief that has clinical utility beyond a particular theoretical or therapeutic model. The described Pathogenic Belief Scale (PBS) is intended to operationalize a transtheoretical understanding of repetitive patterns of emotion-laden beliefs that develop in younger years for adaptive reasons and that continue to influence current internal and external experience of the world. The results of the exploratory factor analysis suggest that the 34-item PBS has good psychometric properties, including sufficient reliability and concurrent validity. The three-factor solution that was obtained by the EFA identified items based on the underlying beliefs that one cannot rely on others, one is undeserving, and that one has a sense of interpersonal guilt. The current three-factor PBS-34 appears to successfully balance the need for parsimony (i.e., a model with the fewest number of factorsFootnote 4) against the need for plausibility (i.e., a model with a sufficient number of factors to adequately account for the correlations among measured variablesFootnote 5). In order to achieve this balance, we tried to reduce the burden of completing the measures by eliminating items that were not indicative of the three factors while maintaining the validity and clinical utility of the three-factor structure. The factor, cannot rely on others, includes beliefs about others not being able or willing to meet our needs, and reflects a subsequent negative view of self. The factor, undeserving, is unrelated to other people and reflects deserving mistreatment and punishment. The factor, interpersonal guilt, describes beliefs about what you should or should not do in relation to family and friends in order to be accepted.
The three-factor model found in the present study is in line with previous research on beliefs that are related to guilt (O'Connor et al, Reference O'Connor, Berry, Weiss, Bush and Sampson1997). In a study based on clinical and nonclinical samples, O'Connor et al. (Reference O'Connor, Berry, Weiss, Bush and Sampson1997) found two distinct factors of guilt-related beliefs: self-hate, which corresponds to our undeserving factor, and composite guilt (survivor guilt, separation guilt, and omnipotent responsibility), which closely corresponds to our interpersonal guilt factor.
Also, in a recent psychometric study on the newly developed Negative Core Belief Inventory in a nonclinical sample (Osmo et al., Reference Osmo, Duran, Wenzel, de Oliveira, Nepomuceno, Madeira and Menezes2018), a theory-driven exploratory three-factor solution was initially tested to evaluate dimensions of negative core beliefs about others and about self, the latter being subdivided into helplessness, unlovability, and worthlessness (cf. Beck, Reference Beck2011). Although this solution accounted for an adequate percentage of the total explained variance (54%), it was not supported by theory. The investigators then proposed a four-factor solution: helplessness/ inferiority, helplessness/vulnerability, unlovability, and worthlessness. There is considerable correspondence between these and two of the PBS factors. Undeserving is related to worthlessness (as well as unlovable) and cannot rely on others is related to helplessness.
Another recent study by Gazzillo et al. (Reference Gazzillo, Gorman, Bush, Silberschatz, Mazza, Faccini and De Luca2017) on the development of a scale of interpersonal guilt, identified four types of pathogenic beliefs that support irrational guilt, (survivor guilt, omnipotent responsibility guilt, self-hate, and separation/disloyalty), which correspond to our factors of undeserving and interpersonal guilt. Moreover, there is research evidence suggesting that beliefs that are related to separation guilt and self-hate (feeling undeserving) are particularly prominent in students that have been identified as pathological gamblers (Locke, Shilkret, Everett, & Petry, Reference Locke, Shilkret, Everett and Petry2013), and beliefs characterized as self-hate play an important role in anorexic adolescents (Berhold & Lock, Reference Berghold and Lock2002).
Other self-report measures of pathogenic beliefs, such as the Dysfunctional Attitudes Scale (Weissman & Beck, Reference Weissman and Beck1978), Irrational Beliefs Inventory (Koopmans, Sanderman, Timmerman, & Emmelkamp, Reference Koopmans, Sanderman, Timmerman and Emmelkamp1994), and other similar measures (for a review see Bridges & Harnish, Reference Bridges and Harnish2010), are tied to specific theories of psychopathology or diagnostic models. The nomothetic PBS is different in that it measures transdiagnostic internalized beliefs that people actually expressed in therapy (i.e., they were clinically derived) without allegiance to specific therapy models and theories. Therefore, although other measures of pathogenic beliefs exist and there will likely be some overlap between those measures and the PBS-34, the PBS-34 is unique in that it was developed to reflect the experiences that are identified by the patients themselves, with the objective of being unbiased by specific therapeutic models and theories. Future studies using the PBS-34 with comparable and conceptually distinct measures are needed to determine its convergent and discriminant validity relative to these other measures. This would allow for a comparison between the two versions of the PBS and, specifically, the possible reduction in discriminant validity between the longer PBS-59 and the PBS-34 short version.
The PBS-34 assesses cognitions that are related to common psychopathology symptoms and diagnoses. As such, this transdiagnostic perspective of psychopathology fits with the research domain criteria that have been proposed recently by the National Institute of Mental Health (Insel et al., Reference Insel, Cuthbert, Garvey, Heinssen, Pine, Quinn and Wang2010). Such a perspective can be especially useful to clinicians who typically focus on the expressed complaints of a particular patient and identify specific cognitions, problems, or symptoms as the target of treatment. The fact that the PBS-34 subscales have been validated in both community and clinical samples adds to its clinical validity in naturalistic psychotherapy settings, where patients who seek treatment may or may not have a formal diagnosis. Future studies could extend the findings of this heterogeneous sample by applying the PBS-34 subscales to different patient populations, including more severe forms of psychopathology such as psychotic disorders.
Moreover, there has been increasing interest in transdiagnostic approaches that describe dimensional models of common mental disorders (e.g., p-factor; Caspi & Moffit, Reference Caspi and Moffitt2018; Caspi et al., Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel and Poulton2014). Caspi et al. (Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel and Poulton2014), for example, emphasize the underlying latent structure of psychopathology by assessing the persistence, co-occurrence, and sequential comorbidity of mental disorders. Therefore, childhood maltreatment may exacerbate the possibility of experiencing any psychiatric disorder at all (Keyes et al., Reference Keyes, Eaton, Krueger, McLaughlin, Wall, Grant and Hasin2012; Lahey et al., Reference Lahey, Applegate, Hakes, Zald, Hariri and Rathouz2012).
Limitations
This psychometric report of the PBS-34 is limited by several factors, the first of which was the limited number of cases that was used to derived the pathogenic beliefs (n = 21). A second limitation was the representativeness of the sample that was used for validation in that most of the participants were highly educated and White. Although this ratio does not represent the diversity of the American population at large, it does (unfortunately) reflect the demographic population that is usually seen in outpatient services. In other words, the reported results are likely generalizable to the majority of American outpatient services, which are visited most frequently by White, middle class, relatively educated individuals who present with low to moderate levels of psychiatric symptoms. Future studies on the PBS-34 would benefit from the explicit inclusion of participants from different racial, educational, and psychiatric backgrounds. Representative community samples might be collected with the help of stratified recruitment procedures, whereby the minority ethnicities are represented according to the percentages in the population. Also, future studies on clinical samples could, for example, specifically include patients from outpatient clinics that are located in areas of cities with more diverse patient populations. The reported initial validation results of the PBS-34 would be strengthened by conducting a confirmatory factor analysis on a separate set of participants to ensure that the factor structure holds.
Another limitation was the use of online recruitment of community participants. The online survey is an efficient, flexible, and cost-effective method (Batterham, Reference Batterham2014), which enabled the recruitment of a relatively large community sample. Although the external validity of online community samples could be debated (Krantz & Dalal, Reference Krantz, Dalal and Birnbaum2000), they have been deemed representative of the general population at large (e.g., Meyerson & Tryon, Reference Meyerson and Tryon2003) and could be considered to provide a normative pattern of association. As in any naturalistic community sample, an unknown percentage of the participants might have suffered from and/or been treated for psychological symptoms (e.g., Olfson, Marcus, Druss, & Pincus, Reference Olfson, Marcus, Druss and Pincus2002). Although the clinical sample of patients with anxiety and/or depressive symptoms reflects common psychopathology in outpatient psychotherapy settings, it would be useful in future research to obtain clearly delineated diagnoses in order to assess the generalizability of our results.
Furthermore, the PBS-34 was developed as a self-report measure. It could be argued that many pathogenic beliefs are at least in part unconscious (Silberschatz, Reference Silberschatz2005), so self-report ratings of pathogenic beliefs only reflect those beliefs that people are consciously aware of or readily able to acknowledge rather than whether those beliefs are present. Previous psychotherapy research (Curtis & Silberschatz, Reference Curtis, Silberschatz and Silberschatz2005; Curtis et al., Reference Curtis, Silberschatz, Sampson and Weiss1994) has shown that clinicians can reliably assess psychotherapy patients’ unconscious pathogenic beliefs. In future research, we plan to compare self-reports and clinician's ratings of pathogenic beliefs to shed further light on possible differences between consciously reported and unconscious pathogenic beliefs.
Similarly, it is important to emphasize that the MOPS is based on retrospectively recalled adverse parenting experiences, rather than actual inadequate parenting. In other words, the perception of that parenting might be distorted (possibly due to these inadequate parenting experiences). However, in their review of the literature, Brewin, Andrews, and Gotlib (Reference Brewin, Andrews and Gotlib1993) concluded that retrospective recollections of adversity, although not perfect, are for the most part quite accurate. Gaining information about parenting experiences in childhood retrospectively has advantages (i.e., immediate and comprehensive information) and has proven to be valid in more longitudinal studies (Wilhelm, Niven, Parker, & Hadzi-Pavlovic, Reference Wilhelm, Niven, Parker and Hadzi-Pavlovic2005).
Implications for Research and Clinical Practice
One of the research advantages of identifying factors in a scale is that it allows for the opportunity to conduct moderation analyses based on these factors in the future. Moreover, a measure like the PBS could potentially be a useful tool in etiological research, perhaps helping investigators to identify cognitive mechanisms that connect trauma and adversity to psychopathology or helping to understand why two individuals that are exposed to the same type of trauma or adversity might manifest very different types of psychiatric symptoms.
The three-factor structure of the PBS-34 may prove to be a useful tool to explain a range of pathology that is seen in outpatient clinics. Future research should explore the relationship between the PBS and various personality and diagnostic instruments. It remains to be seen whether the factor cannot rely on others, for example, has a different meaning for an obsessional patient than for a borderline, narcissistic, paranoid, or delusional patient. It will also be important to determine whether such diagnostic labels exaggerate the homogeneity among groups of patients (Silberschatz, Reference Silberschatz2017). For instance, two depressed patients with the identical DSM diagnosis may have very different underlying pathogenic beliefs that require different therapeutic interventions. A patient whose depression is related to separation or survivor guilt, for example, is not at all similar to a patient whose depression stems from the belief that they cannot rely on others. The PBS-34 subscales could help therapists make these important clinical distinctions early in treatment, thereby allowing for more individualized, patient-centered therapy. Research in psychotherapy has shown that the ability to identify pathogenic beliefs early in treatment and to target interventions that disconfirm specific pathogenic beliefs appears to be an effective ingredient in psychotherapy (Silberschatz, Reference Silberschatz2017). The transtheoretical nature of the PBS means that the risk of overemphasizing certain types of beliefs at the expense of others is minimized. The PBS may be used across different therapies and theoretical orientations, so it could help compare the internalized beliefs of patients in different treatments.
Although all three PBS subscales were significantly related to the different psychopathology measures, it appeared that the subscale, cannot rely on others, was more closely linked to overall symptoms. It is conceivable that people who feel that they cannot rely on other people experience more disruption in their general level of functioning compared with those who feel they do not deserve to be treated well or who experience excessive interpersonal guilt. Both the community and clinical samples completed the PBS and the MOPS. Across the two samples, there were no differences in the correlations between cannot rely on others and undeserving and the MOPS. There was a difference between the correlation between interpersonal guilt and the MOPS in the community sample, where the correlation was not significant, and the clinical sample, where the correlation was significant. The difference that was observed in these data is intriguing, suggesting that beliefs that are related to guilt may be an important consideration in categorizing who will seek therapy and who will not.
In order to highlight further clinical relevance of the PBS-34, it would be interesting to apply several assessments across the course of therapy to better understand the dynamic associations of the PBS-34 subscales with symptoms or presenting complaints. Analyzing the cross-lagged associations of the PBS-34 with outcome measures across therapy could potentially elucidate whether the level of pathogenic beliefs at beginning of therapy or PBS-34 changes across therapy are associated with symptom change. Moreover, future studies should analyze the associations of the PBS with measures of general mechanisms of change in psychotherapy (e.g., the Individual Therapy Process Questionnaire; Mander et al., Reference Mander, Schlarb, Teufel, Keller, Hautzinger, Zipfel and Sammet2015). Besides the use of the PBS-34 as a mediator or moderator of change, it could also be useful in future studies to explore how the PBS-34 and its subscales might be used as a transdiagnostic outcome measure to assess change in psychotherapy. Belsky and Pluess (Reference Belsky and Pluess2009) argue that some people are not just more vulnerable than others to the negative effects of adversity (in line with the prevailing diathesis-stress view of pathology), they are also disproportionately susceptible to the beneficial effects of supportive and enriching experiences, such as those that are offered by the therapeutic environment. As some children appear to be more affected by the quality of care they experience,both negatively and positively, than others (Belsky & Pluess, Reference Belsky and Pluess2009; Leighton et al., Reference Leighton, Botto, Silva, Jiménez and Luyten2017), it is possible that people with certain levels/types of internalized pathogenic beliefs are also better able to make use of psychotherapy than others are.
In conclusion, the PBS-34 is a unique measure of pathogenic beliefs that was designed to reflect experiences that were identified by the patients themselves (bottom-up), rather than by specific therapeutic models and theories. The three-factor solution of cannot rely on others, interpersonal guilt, and undeserving may offer a useful way to categorize transdiagnostic internalized beliefs that are relevant to common psychopathology symptoms. The subscales on the PBS-34 could help delineate and elucidate patients’ pathogenic beliefs that are relevant to the psychotherapy process.
Supplementary Material
The supplementary material for this article can be found at https://doi.org/10.1017/S0954579419001615.
Author contributions
The first author conducted the initial data analyses and developed the original manuscript as well as the later revisions of content and format. The second author added data to the original dataset and conducted further statistical analyses. The third author developed the research idea and the original measure that is described in the paper and revised the document on several occasions. The fourth author was responsible for data collection and data maintenance.
Financial support
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Conflict of interest
The authors have no conflicts of interest to declare.
Ethical standards
All of the procedures that were performed in this study involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all of the participants that were included in the study. The study protocol has been approved by the research institute's committee on human research.
