1. Zero-Order Correlations

The most straightforward analytic method is to examine the correlation between the external variable and each of the individual dimensions. In the presence of a DGF, this approach is problematic because all of the observed correlations reflect a composite of general and specific factor variance. Thus, a large correlation could mean that (a) the general factor influences the criterion, (b) the specific factor does, or (c) both do. Conversely, a correlation of zero could reflect that (d) neither the general nor the specific factors are related to the criterion or (e) both are, but in opposite directions. In short, zero-order correlations cannot separate general and specific factor influences, rendering interpretation ambiguous at best. Nevertheless, researchers frequently interpret dimension scale correlations as though they reflect only specific factor variance (e.g., Kinicki et al., Reference Kinicki, McKee-Ryan, Schriesheim and Carson2002).

A related issue occurs when using a composite to index the general factor. Such a sum score reflects not only the general factor but also specific factors and measurement error. As estimates of general factor influence, composite score correlations will be inflated if the general and specific factor(s) predict in the same direction (i.e., enhancing conflation) or attenuated if they predict in opposite directions (i.e., suppressive conflation). Composite correlations reflect an average criterion relation across dimensions, not simply the effect of the general factor.

On the basis of zero-order correlations from Edwards et al. (Reference Edwards, Bell, Arthur and Decuir2008), researchers would conclude that overall satisfaction and satisfaction with work itself have weak positive relations to task performance but that other scales show negligible relations (see Table 1). Similarly, one would conclude that contextual performance is weakly to moderately positively related to overall, work, pay, and supervisor satisfaction but not satisfaction with promotions or coworkers. Finally, on the basis of correlations with a satisfaction composite, researchers would conclude that overall satisfaction is positively related to contextual performance but not task performance. As the analyses below show, most of these conclusions would be wrong.

2. Multiple Regression

Multiple regression examines the combined predictive effects of dimension scores for a criterion of interest. Analyses may be performed either with observed scale scores or with latent variables in an SEM framework (but without specifying a general factor). In either case, this approach is problematic in the presence of a DGF because the influence of the general factor will create multicollinearity problems, making the pattern of regression or structural path coefficients unstable. Further, regression coefficients will primarily reflect each dimension scale's loading on the DGF, not their unique influence on the criterion. In the case of suppressive conflation, R ² for the model will also be an underestimate, as positive and negative effects of the general and specific factors cancel out within the individual scales. Although multiple regression is affected by intercorrelations among predictors, researchers often focus their interpretations on specific factors rather than attending to the influence of the general factor (e.g., Edwards et al., Reference Edwards, Bell, Arthur and Decuir2008). From the multiple regression results in Table 1, researchers would come to similar conclusions as earlier: Overall satisfaction and work satisfaction are positively related to task performance, whereas overall satisfaction and supervisor satisfaction are positively related to contextual performance. In addition, satisfaction with promotions and coworkers shows small negative effects, suggesting that once other facets are controlled, these dimensions are negatively related to the criteria. R ² values show that as a set, satisfaction measures explain only a moderate amount of variance in task and contextual performance.

3. General Factor SEM

A third method for assessing relations to external criteria focuses entirely on the DGF. That is, general factor SEMs attribute all predictive power to the general factor; specific factor variance is ignored entirely. Although ostensibly relevant in the presence of a DGF, this approach is essentially the same as using a sum score composite and shares many of its disadvantages. In the case of enhancing conflation, dimension loadings on the general factor and the structural path from the general factor to the criterion will be inflated. In the case of suppressive conflation, the same process will attenuate the general factor's structural coefficient. In both cases, specific effects are forced through the general factor to fit the model. Consequences of failing to specify specific factor structural paths are most severe when general factor saturation is weak. Because a general factor SEM is the latent analogue to an observed score composite, it is unsurprising that their results are virtually identical (see Table 1).

4. Bifactor Model

The preceding analytic approaches share a common limitation: General and specific factor variances are not disentangled in the predictive model, biasing the conclusions drawn about the DGF and specific factors. Bifactor modeling offers a solution (Reise, Reference Reise2012). In a bifactor model, each indicator loads on the general factor (which influences all measures) and a specific factor (which influences some measures). General and specific factors are constrained to be uncorrelated, which allows the unique predictive power of each to be examined separately. A bifactor model of Edwards et al.'s (Reference Edwards, Bell, Arthur and Decuir2008) data is shown in Figure 1.

Figure 1.

Standardized loadings for a bifactor model of the Edwards et al. (Reference Edwards, Bell, Arthur and Decuir2008) job satisfaction measures with 95% bootstrapped confidence intervals.

Bifactor predictive models are best estimated via SEM and can be fit using a variety of software packages (for an alternative approach using multiple regression and residualized factor scores, see Salgado, Moscoso, & Berges, Reference Salgado, Moscoso and Berges2013). Before examining how a bifactor model affects interpretations of DGFs, a few words about how bifactor modeling works are in order. In fitting a bifactor SEM, ideally, multiple indicators for each specific factor are used (e.g., multiple items from each scale of a personality measure; McAbee, Oswald, & Connelly, Reference McAbee, Oswald and Connelly2014). Using multiple indicators allows specific factors to represent shared reliable variance, rather than a mix of reliable and error variance, and permits their simultaneous inclusion in the predictive model (cf. McAbee et al., Reference McAbee, Oswald and Connelly2014). However, if single indicators are used for specific factors, such as when only scale scores are available or when reanalyzing a published correlation matrix, at least one specific factor must be excluded from the predictive model to avoid exact linear dependence (Chen, Hayes, Carver, Laurenceau, & Zhang, Reference Chen, Hayes, Carver, Laurenceau and Zhang2012). Thus, when predicting performance using a bifactor model of Edwards et al.'s (Reference Edwards, Bell, Arthur and Decuir2008) data, we excluded the uniqueness of the single “overall satisfaction” item, based on the assumption that most of this variance was error (Wanous, Reichers, & Hudy, Reference Wanous, Reichers and Hudy1997).

Results of the bifactor analyses produce a striking pattern (see Table 1). After removing the general factor variance, all specific satisfaction factors are negatively correlated with both criteria, although they vary widely in their magnitude and precision. What this means is that the positive relation to performance comes from the DGF, overall satisfaction, not from evaluations of particular work features. For purposes of contrast, such a conclusion is precisely the opposite of that of Kinicki et al. (Reference Kinicki, McKee-Ryan, Schriesheim and Carson2002). Further, overall, promotions and coworkers satisfaction factors show far stronger relations with performance than were observed in any of the analyses wherein their variances were conflated, illustrating the relevant, but hidden, effects of suppressive conflation. Such a pattern of results is similar to bifactor analyses of other constructs (cf. Chen et al., Reference Chen, Hayes, Carver, Laurenceau and Zhang2012).

Despite their interpretive advantages, bifactor analyses can present several challenges. Two of these are worth mentioning. First, in some cases, one or more specific factors may have negligible or negative estimated variances and factor loadings, indicating that the specific factor is inseparable from the general factor. In such situations, the offending specific factors should be eliminated and their indicators allowed to load only onto the general factor. Second, like any SEM, bifactor models require sufficiently large sample sizes to provide stable parameter estimates. Sample requirements depend on the degree of communality in the indicators and factor overdetermination (i.e., the degree to which each factor shows strong loadings on multiple indicators; MacCallum, Widaman, Zhang, & Hong, Reference MacCallum, Widaman, Zhang and Hong1999). A concern in bifactor models is that if DGF saturation is large, specific factor loadings will be too weak to provide stable estimates of external relations without very large sample sizes. In such cases, impact of the apparently minor specific factors would not be large enough to justify the costs of trying to measure them reliably.