Freudenberger (Reference Freudenberger1974) coined the term burnout as a state of mental and physical exhaustion caused by worker’s professional life. Thus, individuals who burn out from their work would deplete their energetic resources and lose their dedication to work (Bakker et al., Reference Bakker, Demerouti and Sanz-Vergel2014). Burnout was initially thought to describe human service workers’ feeling of emotional exhaustion, depersonalization, and reduced personal accomplishment (Taris, Schreurs, & Schaufeli, Reference Taris, Schreurs and Schaufeli1999). However, the concept has now expanded to all workers who interact with people to some extent. Accordingly, burnout may be defined as a syndrome characterized by emotional exhaustion, depersonalization and lack of personal accomplishment (Bakker et al., Reference Bakker, Demerouti and Sanz-Vergel2014).
On the other hand, work engagement may be seen as a natural criterion for the validity of burnout factors, as work engagement is defined as a positive, accomplished, and encouraged attitude towards the job. According to this, Maslach and Leiter (Reference Maslach and Leiter1997) stated that work engagement may be conceptualized as the opposite pole of burnout. Indeed, there are two theoretical positions on the nature of both constructs. One side argues that both burnout and work engagement are two opposite poles of the same continuum, while others consider the construct to be an independent concept, that shows a negative relationship with burnout. On one hand, Maslach and Leiter (Reference Maslach and Leiter1997) considered burnout and work engagement as two parts of the same composition. According to these authors, burnout would be composed by emotional exhaustion, cynicism, and reduced professional efficacy, while work engagement would be defined by vigor, involvement and efficacy. On the other hand, Schaufeli and Bakker (Reference Schaufeli and Bakker2001) defended that burnout and work engagement constituted two different concepts.
Emotional exhaustion relates to being emotionally depleted of your emotional resources (Taris et al., Reference Taris, Schreurs and Schaufeli1999). Depersonalization or cynicism refers to “a negative or excessively detached response toward these people, who are the recipients of one’s service or care” (Bakker et al., Reference Bakker, Demerouti and Sanz-Vergel2014, p. 390). Finally, the third dimension referred to a reduced personal accomplishment, can be defined as the “tendency to evaluate one’s work negatively. Such an evaluation is often accompanied by feelings of insufficiency, self-doubt and poor self-esteem” (Xanthopoulou, Bakker, Demerouti, & Kantas, Reference Xanthopoulou, Bakker, Demerouti and Kantas2012, p. 41).
Several measures or instruments for the measurement of burnout have been developed. With no intention to be exhaustive, the Oldenburg Burnout Inventory (OLBI; Demerouti & Nachreiner, Reference Demerouti and Nachreiner1998), the Staff Burnout Scale for Health Professionals (Jones, Reference Jones1980), the Burnout Measure (Malakh-Pines & Aronson, Reference Malakh-Pines and Aronson1988), the Shirom-Melamed Burnout Measure (SMBM; Shirom & Melamed, Reference Shirom and Melamed2006), and the Copenhagen Burnout Inventory (Kristensen, Borritz, Villadsen, & Christensen, Reference Kristensen, Borritz, Villadsen and Christensen2005). Among them, the OLBI covers both affective aspects of exhaustion, and also physical and cognitive aspects (Demerouti, Bakker, Vardakou, & Kantas, Reference Demerouti, Bakker, Vardakou and Kantas2003). Analyses on German employees revealed a two correlated factor structure: exhaustion and disengagement. Another recently developed scale is the Copenhagen Burnout Inventory, which focuses on exhaustion as the most important element of burnout. This instrument is integrated by three scales: personal burnout, work-related burnout, and client-related burnout (Borritz et al., Reference Borritz, Rugulies, Bjorner, Villadsen, Mikkelsen and Kristensen2006). There are also some measures originally developed for Spanish-speaking populations. For example, the Spanish Burnout Inventory is composed by four sub-scales, namely enthusiasm toward the job, psychological exhaustion, indolence, and feeling guilty (Gil-Monte & Zúñiga-Caballero, Reference Gil-Monte and Zúñiga-Caballero2009).
However, the most widely used scale for measuring the components of burnout is, by far, the Maslach Burnout Inventory in its different versions: MBI-Human Services Survey, MBI-Educators Survey, MBI-General Survey, and MBI-Student Survey (Maslach, Jackson, & Leiter, Reference Maslach, Jackson and Leiter1996; Schaufeli, Martinez, Pinto, Salanova, & Bakker, Reference Schaufeli, Martinez, Pinto, Salanova and Bakker2002). Among the available versions of the scale, the Maslach Burnout Inventory-General Survey (MBI-GS; Schaufeli, Leiter, Maslach, & Jackson, Reference Schaufeli, Leiter, Maslach, Jackson, Maslach, Jackson and Leiter1996) was developed to measure three dimensions of burnout (cynicism, personal accomplishment, and emotional exhaustion) regardless of the type of work. It is composed of 16 items, five of them corresponding to cynicism, five others resembling emotional exhaustion, and six assessing personal accomplishment. The MBI-GS differs from the original MBI in that the former includes a more general view about the aspects of the job, which is not restricted to their social features; nevertheless, the sub-scales resemble those of the original form.
Although being an extremely popular measure of burnout, the MBI in its different versions is not free of criticism for a number of reasons. For the particular case of the MBI-GS, one of them is that the MBI-GS presents the same psychometric limitations as its original version “namely, that the items in each subscale are all framed in the same direction” (Demerouti et al., Reference Demerouti, Bakker, Vardakou and Kantas2003, p. 297). This implies that the “negative” factors of the scale (cynicism and emotional exhaustion) are composed by negatively worded items, and by contrast, the positive factor, or personal accomplishment, is positively worded. Also, the Maslach Burnout Inventory has raised some doubts on its validity, as there has been disagreement regarding the number of factors that make up the construct. Some authors have suggested that the factor structure for the MBI is not stable across occupational groups (Leiter & Schaufeli, Reference Leiter and Schaufeli1996). In particular, the depersonalization and emotional exhaustion subscales tended to collapse into one factor when groups other than human service providers completed the MBI (Bakker, Demerouti, & Schaufeli, Reference Bakker, Demerouti and Schaufeli2002). That was indeed one reason to develop the MBI-GS. The MBI-GS assumes the three-factorial hypothesis by which emotional exhaustion and cynicism correlate and personal accomplishment is shown as an independent factor. Additionally, there are two other factorial models that try to explain the concept of burnout and which can be found along literature. The first of them hypothesizes a unidimensional structure of burnout (Malakh-Pines & Aronson, Reference Malakh-Pines and Aronson1988). Green, Walkey, and Taylor (Reference Green, Walkey and Taylor1991) found a two-factor structure with one dimension formed by emotional exhaustion and cynicism, and the other conformed by personal accomplishment.
The MBI-GS is a relatively young instrument and thus there is limited literature about its validations. However, it is worth noting that the measure has been validated in Canada (Leiter & Schaufeli, Reference Leiter and Schaufeli1996), in Sweden (Schutte, Toppinen, Kalimo, & Schaufeli, Reference Schutte, Toppinen, Kalimo and Schaufeli2000), in the Netherlands (Bakker et al., Reference Bakker, Demerouti and Schaufeli2002), in Japan (Kitaoka-Higashiguchi, Ogino, & Masuda, Reference Kitaoka-Higashiguchi, Ogino and Masuda2004), in Finland (Ahola et al., Reference Ahola, Honkonen, Isometsä, Kalimo, Nykyri, Koskinen and Lönnqvist2006), in Norway (Langballe, Falkum, Innstrand, & Aasland, Reference Langballe, Falkum, Innstrand and Aasland2006), and in Poland (Chirkowska-Smolak & Kleka, Reference Chirkowska-Smolak and Kleka2011).
With respect to the Spanish version of the scale, the instrument has been validated in Spain (Gil-Monte, Reference Gil-Monte2002; Moreno-Jiménez, Rodríguez-Carvajal, & Escobar, Reference Moreno-Jiménez, Rodríguez-Carvajal and Escobar Redonda2001), in Cuba (Oramas, González, & Vergara, Reference Oramas, González and Vergara2007), and Venezuela (Millan & d’Aubeterre, Reference Millan and d’Aubeterre2012). These four validations studied the factor structure by exploratory means, most of them with orthogonal rotations (i.e., uncorrelated factors) and all of them supported the theoretical three-factor solution. A confirmatory study on the Spanish version of the MBI-GS by Salanova, Schaufeli, Llorens, Peiró, and Grau (Reference Salanova, Schaufeli, Llorens, Peiró and Grau2000) found some evidence for a three factor structure. Nevertheless, this research presented some shortcomings: an incidental sample; the use of a estimation method (Maximum Likelihood) that assumes multivariate normality and continuous variables in a questionnaire with ordinal variables usually non-normal; and more important, the original three factor structure did not fit well, and in order to improve model fit four error covariances were introduced. Covarying covariances has been extensively criticized as having very little theoretical basis and inflating the fit (for example, Kline, Reference Kline2011).
All in all, the MBI has been subjected to extensive research and has a well-documented history of validity and reliability studies. Nevertheless, the MBI-GS has not been so widely validated, as seen in the aforementioned literature. This is even clearer with the Spanish version of the MBI-GS. First, the validation studies have been exploratory, with no theoretical reasons to use principal component analysis and orthogonal rotations, instead of correlated factors as defended by the authors of the scale. The available confirmatory results in a Spanish sample (Salanova et al., Reference Salanova, Schaufeli, Llorens, Peiró and Grau2000) did not fit the data well until four error covariances were included. All this makes it extremely difficult to compare the relative fit of the different structures proposed in the literature. Second, most of the validations have used non-probabilistic samples. Additionally, and to the knowledge of these researchers, the MBI-GS has never been validated in the Dominican Republic. Therefore, the aim of this research is to offer evidence on the psychometric properties of the MBI-GS, specifically reliability, factorial and criterion-related validities, for its use in the Dominican Republic and potentially other Caribbean Spanish-speaking countries, using a large and representative sample of Dominican teachers, as well as offering normative data to be used with these professionals.
Method
Sample, design and procedure
The sample used in this study comes from a research project financed by the Ministerio de Educación of the Dominican Republic, whose interests included to measure burnout and related topics in teachers. The sample consists of currently-employed teachers in the Dominican Republic. Stratified sampling was applied with proportional allocation and simple random sampling within each stratum. Sample size from each stratum was proportional to stratum size in relation to the total population. Stratums were the Educational Districts and the sample size within each stratum was proportional to their proportions in the total population. In order to obtain the sample size, the margin of error and confidence level were set at 3% and 95%, respectively. The result was a potential sample size of 931. Additionally, up to 5% of the sample was predicted not to attend the data-gathering process, and accordingly sample size was proportionally increased. Thus, the final sample size was 978 (931*1.05). The sample was composed by 73.65% women and the remaining 26.35% men. Teachers were mostly involved in basic level teaching (54.4%), with an additional substantial proportion of the sample being teachers of middle level (18.1%). The rest of the sample (27.5%) were initial level teachers, special education teachers, adult’s and young people’s teachers, headmasters, assistant headmasters, scholar psychologists and other. Their average age was 42.9 years old, with a standard deviation of 8.7 years old, and a range between 23 and 76 years. Data gathering was made during the first semester 2015. The questionnaires were administrated by trained interviewers working for Ministerio de Educación of the Dominican Republic. No reward was given for completion.
Instruments
The survey included a number of different job-related questionnaires, as well as socio-demographic information. For the purposes of this research two scales were used:
-
a) Maslach Burnout Inventory-General Survey (Schaufeli et al., Reference Schaufeli, Leiter, Maslach, Jackson, Maslach, Jackson and Leiter1996). The MBI-GS is a 16-item instrument and the scale goes from 1 (totally disagree) to 5 (totally agrees). Theoretically, five items correspond to cynicism (2, 5, 8, 11, 14), five others measure emotional exhaustion (1, 4, 7, 10, 13), and six conform to professional efficacy (3, 6, 9, 12, 15, 16).
-
b) The Utrecht Work Engagement Scale (Schaufeli & Bakker, Reference Schaufeli and Bakker2003). This scale is composed of 17 items tapping three dimensions: vigor (1, 4, 8, 12, 15, 17), dedication (2, 5, 7, 10, 13), and absorption (3, 6, 9, 11, 14, 16). The scale consists of five categories, ranging from 1 (completely disagree) to 5 (completely agree). Internal consistencies as estimated with the Composite Reliability Index (CRI; Raykov, Reference Raykov2004) were .85 for vigor, .85 for dedication, and .74 for absorption. The confirmatory factor analysis indeed supported a three-factor structure with the following fit indices and statistics: χ2(116) = 769.64, p < .001, RMSEA = .076 (.071 – .081) and CFI = .958.
Statistical analyses
The factorial validity was studied through confirmatory factor analysis. Several competing models, those found in the literature, were proved in order to test the dimensionality of the scale. The estimation method employed in the CFAs was Weighted Least Squares, specifically both WLM and WLSMV estimation methods. These methods have been shown to perform very well in the case of ordinal (less than seven categories) and non-normal data (Asparouhov & Muthén, Reference Asparouhov and Muthén2010; Finney & DiStefano, Reference Finney, DiStefano, Hancock and Mueller2013; Flora & Curran, Reference Flora and Curran2004; Muthén, du Toit, & Spisic, Reference Muthén, du Toit and Spisic1997; Rhemtulla, Brosseau-Liard, & Savalei, Reference Rhemtulla, Brosseau-Liard and Savalei2012). The models were: a) a one factor model of burnout underlying all items in the MBI-GS; b) a two-factor model with depersonalization and emotional exhaustion collapsed into a single factor (core burnout) as proposed among others by Bakker et al. (Reference Bakker, Demerouti and Schaufeli2002) or Green et al. (1991); c) the original three factor structure (Schaufeli et al., Reference Schaufeli, Leiter, Maslach, Jackson, Maslach, Jackson and Leiter1996); d) the three factor structure after removing item 11 due to its poor reliability. As recommended in the literature (Kline, Reference Kline2011) several fit indices were used: the chi-square statistic (χ2), the comparative fit index (CFI), and the root mean squared error of approximation (RMSEA). Weighted Least Squares Mean and Variance corrected was used as the method of estimation, the best performing method for non-normal and ordinal data (Kline, Reference Kline2011). Generally, it is assumed that a CFI higher or equal to .90 together with a measure of amount of error such as the RMSEA lower than .08 are considered adequate, with CFI > .95 and RMSEA < .05 indicating excellent fit (Hu & Bentler, Reference Hu and Bentler1999). Additionally a weighted version of the RMR, the WRMR, has been calculated. This index has been proposed when WLS is used as the estimation method. It has not yet well-stablished cut-off criteria, but it may be useful for model comparison. The included indices are those available when WLS is used. Statistical analyses also included estimations of internal consistency (Cronbach’s alpha and composite reliability index, CRI) and criterion-related validity relating work engagement and burnout. These analyses were conducted with SPSS 21 and MPLUS 6.
Results
Factorial Validity
The four models shown in the Figure 1 were examined in order to test their validity. The first three models are completely a priori, and correspond to: Model 1, a unidimensional measure of burnout; Model 2, a two-factor structure of burnout (professional efficacy and the core of burnout); and Model 3, the theoretical three-factor structure (professional efficacy, cynicism, and emotional exhaustion). Finally, a fourth model was tested which is a slight ad hoc modification of model 3 that removes item 11 (an indicator of cynicism) given its low factor loading on the intended factor.
Alternative confirmatory factor models
Table 1 shows the fit indices for the aforementioned four models. Best fitting model among the a priori ones was model 3. Model 3 corresponds to the theoretical structure of the MBI-GS. Nevertheless, item 11 had an extremely low factor loading on the cynicism dimension (λ = –.13) and was therefore removed. That yield to model 4, which was in turn the best fitting model. Standardized and unstandardized factor loadings for model 4, the one retained, are presented in Table 2.
Goodness-of-fit indices for the tested models
Note: **p < .01.
Unstandardized and standardized factor loadings for the best-fitting model
Note: **p < .01.
Internal consistency
Cronbach’s alphas for the three-factor structure found via CFA for the Spanish version of the MBI-GS were .67 for emotional exhaustion, .77 for professional efficacy and .48 for cynicism. When item 20 was removed from the cynicism dimension, its alpha estimate increased up to .54. As well as alpha, CRI’s were also calculated, and results were .78 for emotional exhaustion, .86 for professional efficacy, and .65 for cynicism. When item 11 was removed from the cynicism dimension, its internal consistency increased up to .74. According to the CRI, internal consistency seems adequate. Once dimensionality and internal consistencies of those dimensions is shown to be adequate, norms can be established, given that the sample is representative of the overall population of teachers in the Dominican Republic. These norms are presented in Annex I.
Criterion-related validity
Relations among job burnout and work engagement provide evidence of criterion-related validity, as both constructs have been hypothesized as opposite poles or, at least, as highly related. Table 3 shows the correlations among the dimensions of burnout (emotional exhaustion cynicism, and professional efficacy) and the three dimensions of work engagement (vigor, dedication, and absorption). As expected, the correlations among the dimensions of burnout and those of work engagement were statistically significant and in the expected direction.
Correlations among the dimensions of job burnout and work engagement
* p < .05; **p < .01.
Discussion
The aim of this work was to offer evidence on the reliability and validity of the most widely used measure of burnout for its use in the Dominican Republic and potentially other Spanish-speaking Caribbean countries. Additionally, this research is, to the knowledge of these researchers, the first confirmatory validation of the MBI-GS performed on a Spanish-speaking sample. Specifically, the study taps the factorial and criterion-related validity of the scale and its internal consistency. The added value of this research is twofold: first the representative (probabilistic) sample used is scarcely employed in other validity studies and allows to present normative data for the population under study; second, studying a sample of Dominicans, a Spanish-speaking population with some that presents language variations from Spain and other South American countries offers further evidence on the generalizability of psychometric properties of the MBI-GS.
With respect to factorial validity, among all models proposed in the literature to account for variability in its indicators, it was found that the best fitting model was the three-factor structure composed by cynicism, emotional exhaustion and professional efficacy. This parallels the original MBI structure (Maslach & Jackson, Reference Maslach and Jackson1986) and provides further evidence of the polymorphic nature of the burnout construct. However, the factor loading of item 11, an indicator of the cynicism dimension, suggested reconsideration. After suppression of this item, the three-factorial structure showed a considerably improved fit. Of course, the empirical problem with the psychometric properties of item 11 could be related to the particularities of the variant of Spanish spoken in the Dominican Republic. However, this does not seem to be the case as two Dominicans experts in organizational psychology reviewed the adaptation and they did not point out at any particular difficulty with this item. It could also be a statistically particular result to the studied population. If this was the case, then further research in similar populations should clarify the behavior of this indicator. It is worth noting that a confirmatory research on the factorial validity of the MBI-GS in five occupational groups from three different countries (Schutte et al., Reference Schutte, Toppinen, Kalimo and Schaufeli2000) also found that cynicism had less stable and reliable indicators compared the other dimensions, and sometimes removing items of cynicism was needed to achieve an adequate fit. Moreover, Chirkowska-Smolak and Kleka (Reference Chirkowska-Smolak and Kleka2011), also removed an item of cynicism in order to obtain good model fit. The cynicism indicator removed in these two studies was the same one, but a different one from ours. Nevertheless, these results taken together suggest some minimal difficulties with the cynicism dimension. Besides this specificity, the basic three-factor structure of the MBI-GS was strongly supported by our results. This is in line with the dimensionality defended and found by the authors for the MBI-GS (Schaufeli et al., Reference Schaufeli, Leiter, Maslach, Jackson, Maslach, Jackson and Leiter1996) and other versions of the MBI, as the MBI-Human Services Survey, the MBI Student Survey, or the MBI- Educators Survey (Maslach et al., Reference Maslach, Jackson and Leiter1996; Schaufeli, Salanova, González-Romá, & Bakker, Reference Schaufeli, Salanova, González-Romá and Bakker2002). Contrarily, our results are neither consistent with the unidimensional model found by Malakh-Pines and Aronson (Reference Malakh-Pines and Aronson1988) nor with the two-factor structure supported in Green et al. (Reference Green, Walkey and Taylor1991). It is also true that the one and two-factor models just mentioned have found far less support in the literature than the three-factor model. The factor structure results are also of interest because better estimation method for the type of data at hand has been used, as stated in the method section.
Regarding internal consistency, results found for the three-factor structure of the MBI-GS as compared to those obtained for other Spanish-speaking samples, show relative disparities. For example, Oramas et al. (Reference Oramas, González and Vergara2007) found alphas of .70, .80, and .78 for emotional exhaustion, professional efficacy and cynicism, respectively. Likewise, differences between these results and results obtained in other general samples are even more dissimilar. For example, Schutte et al. (2000) found alphas of .86, .83 and .75 for emotional exhaustion, professional efficacy and cynicism, respectively. In spite of these differences, it should be noted that these results only suggest that internal consistencies, as measured by alpha, are inadequate. The results do not suggest that internal consistency of the Spanish version applied in the Dominican Republic is inadequate, because the CRI´s are well above the cut-off criteria of .7. Alpha is the most widely used estimator of internal consistency of tests, but it is also well-known that it has several disadvantages, among others, it has been criticized as being only completely appropriate with essentially tau-equivalent tests, and by being a lower bound for the true reliability A tau-equivalent test assumes all items measure the same latent variable, on the same scale, with the same degree of precision, with all true scores being equal, conditions that do not hold for this particular scale (indeed rarely do so). When tau-equivalence does not hold, alpha would normally underestimate the population value. An alternative to coefficient alpha is the Composite Reliability Index, calculated using the results from confirmatory factor analyses.
Concerning criterion-related validity, the three factors were correlated in the expected direction. Professional efficacy, a dimension of burnout measured in the opposite direction adequately correlated positively with the three factors of work engagement. The amount of correlation of professional efficacy with these three criteria was extremely similar to those found, for example by Xanthopoulou et al. (Reference Xanthopoulou, Bakker, Demerouti and Kantas2012), .65, .70, and .53, respectively for vigor, dedication and absorption. Nonetheless, emotional exhaustion and cynicism negatively correlated with the three dimensions of work engagement, as expected, but in a much lesser amount than found in Xanthopoulou et al. (Reference Xanthopoulou, Bakker, Demerouti and Kantas2012), who found negative correlations ranging from –.32 to –.71, whereas the correlations for this study ranged from a minimum of –.06 to a maximum of –.29. These results indicate a greater discriminant validity of emotional exhaustion and cynicism for teachers in the Dominican Republic than in other populations. Anyway, in general current results offer evidence of adequate criterion-related validity for the MBI-GS.
This research has both strengths and limitations. Among the formers, this is the first study on the psychometric properties of the MBI-GS Spanish version carried out by means of the most adequate estimation methods for confirmatory analyses of ordinal data. It is the first validation on a sample from the Dominican Republic. Moreover, it is the first study on the validity of the Spanish version of the MBI that uses a probabilistic sampling scheme that guarantees sample representativeness. Lastly, given the representative nature of the sample, as well as the novel use of the instrument in the Dominican Republic, some practical implications can be derived from the study. One such implication is the generalizability of the psychometric properties of the MBI-GS to the general population of Dominican teachers, which implies that the use of the instrument is suitable for this specific population. Another implication signals the importance of burnout as well as work engagement in the literature, thus making the validation of the MBI-GS a valuable contribution to this line of research in the Dominican Republic, given that this instrument is the most used measure of burnout. Additionally, further generalizability can be made in that this is the first validation of the MBI-GS in a Spanish-speaking Caribbean country; given the similarities in the language as well as some educational practices among these countries, the results of this study can be interpreted as pointing to good psychometric properties of the instrument in other, similar, countries. However, the study also has some limitations. No estimates of test-retest reliability could be provided due to the cross-sectional nature of the design. Another limitation is that the only construct measured to establish the criterion-related validity was job engagement. An additional limitation is that a post-hoc modification was needed in order to achieve a good fit (to eliminate item 11) and such modifications need further scrutiny with new samples from the same population. Also, regarding the MBI-GS, it is a scale composed by 16 items in which high reliability is hard to achieve. These reliabilities are usually needed for precise estimates for individual assessment which limits the scope of our results. And lastly, given that this is the first validation carried out in a Spanish-speaking Caribbean country the comparison of the results can hardly be made, and therefore further research on this type of populations is needed.
To put the whole thing in a nutshell, factorial results of the MBI-GS offer evidence for a three-factor structure of burnout that includes emotional exhaustion, professional efficacy and cynicism. These three dimensions of burnout, as measured with the MBI-GS, were shown to have adequate levels of validity and reliability.
ANNEX I. Descriptive and normative data for the three dimensions of burnout for total sample, women and men.
