In their article, Ree, Carretta, and Teachout (Reference Ree, Carretta and Teachout2015) argued that a dominant general factor (DGF) is present in most, if not all, psychological measures (e.g., personality, leadership, attitudes, skills). A DGF, according to Ree et al., is identified by two characteristics. First, the DGF accounts for the largest amount of a measure's systematic variance, and second, it influences every subdimension within the construct domain. They indicate that researchers ignore DGFs and pay inappropriate amounts of attention to the specific dimensions (DSs) even though the DGF provides most of the predictive power and the DS adds little predictive power.
In this article, we raise three issues with Ree et al.’s arguments. First, we argue that the evidence for the DGF, at least in one content area (i.e., personality), is not clean cut. Second, we will show that even if there is a DGF in personality measures, the contribution of specific personality dimensions becomes the more dominant source in the measure once monomethod/mono-operational biases are accounted for in the data. Third, we argue that, in contrast to Ree et al., there is growing evidence that DSs in the very domain that originated the DGF debate, cognitive ability, are useful and have predictive power.
Universal DGF? The Case of Personality
Is there really a personality DGF? Although there are publications supporting this claim, this idea is still actively debated (e.g., Ashton, Lee, Goldberg & De Vries, Reference Ashton, Lee, Goldberg and de Vries2009; Hopwood, Wright, & Donnellan, Reference Hopwood, Wright and Donnellan2011), and there is counterevidence against a personality DGF. It is important to recognize these counterarguments to prevent a false impression that a DGF in the personality literature is widely accepted.
Examples of counterevidence to a DGF include Donnellan, Hopwood, and Wright's (Reference Donnellan, Hopwood and Wright2012) attempt to replicate the Rushton and Irwing (Reference Rushton and Irwing2008) study that supported a personality DGF. Using a new sample, Donnellan et al. (Reference Donnellan, Hopwood and Wright2012) failed to find support for a DGF model. Although failing to replicate findings may be a sign of our times, it is more disturbing that Donnellan et al. couldn't replicate the published DGF model when they recreated Rushton and Irwing's variance/covariance matrix using the published correlation matrix combined with variable standard deviations. Donnellan et al. report that they could recapture the DGF model's published degrees of freedom only by adding additional constraints not reported in the original article. Surprisingly, even with these constraints, the DGF model would not converge with Rushton and Irwing variance/covariance matrix nor with their own sample.
Rather than a DGF, many studies found two higher order factors (e.g., DeYoung, Peterson, & Higgins, Reference DeYoung, Peterson and Higgins2002; Digman, Reference Digman1997). Agreeableness, Conscientiousness, and Emotional Stability loaded on one higher order factor called the alpha or stability metatrait factor. Extraversion and Openness to Experience loaded on another higher order factor called the beta or plasticity metatrait factor (DeYoung et al., Reference DeYoung, Peterson and Higgins2002; Digman, Reference Digman1997). Using Ree et al.’s definitions, these metatrait factors are group factors, not DGFs, because each metatrait factor influences only some of the personality dimensions. The question now appears to be not whether the Big Five load onto a single DGF but whether the two aforementioned group factors are sufficiently correlated to justify a third level DGF. The jury is still out because the correlations between the two group factors have ranged from being uncorrelated (DeYoung, Reference DeYoung2006) to being values between .18 and .48 (DeYoung, Peterson, & Higgins, Reference DeYoung, Peterson and Higgins2002; Musek, Reference Musek2007). The smaller the correlation between the two group factors, the less likely it is that a DGF exists.
DeYoung (Reference DeYoung2006) found that smaller group factor correlations are obtained when the variance associated with different types of data sources (e.g., self-ratings, spouses, friends) is incorporated in the statistical analyses. Specifically, higher correlations between the alpha and beta group factors were obtained when he performed statistical analyses separately for each data source (i.e., analysis done for only self-ratings; another analysis performed using friend data). Interestingly, the two group factors were uncorrelated when he reanalyzed the data and incorporated all data sources explicitly in his analyses.
In summary, declaring that there is a DGF among personality dimensions is premature. This may also be true with the other domains covered in Ree et al.
Dominance of Specific Dimensions in Measures
Even if there is a personality DGF, the DSs can still be the dominant source of influence in personality measures. To illustrate how this could be true, we followed the psychometric and mathematical logic outlined by Kuncel and Sackett (Reference Kuncel and Sackett2014) when they discussed assessment center ratings. First, let us assume that we have a Big Five instrument and that we want to create a single personality score for each individual by adding the dimension score ratings together. This would be reasonable to do if one believed in a personality DGF.
As with any measurement instrument, there are three sources of variance that affect personality scores: (a) personality itself, (b) method bias, and (c) random error. The first two variance sources are systematic and can be split into a general and a specific variance portion. This means that we are assuming there is a general personality (i.e., DGF) variance source and a source of variance attributable to specific personality dimensions (DSs). Similarly, we assume that there is a general method variance portion (method general; MG) due to similar characteristics across all methods and a method specific (MS) variance portion that is unique to each method.
Following the findings of Rushton, Bons, and Hur (Reference Rushton, Bons and Hur2008), we set the percentage of variance attributable to personality (both general and specific) to be 58%. We used Mount, Barrick, Scullen, and Rounds's (Reference Mount, Barrick, Scullen and Rounds2005) reliability-corrected correlations among the Big Five dimensions to estimate the average Big Five correlation to be .29. According to psychometric theory (Ghiselli, Campbell, & Zedeck, Reference Ghiselli, Campbell and Zedeck1981), the correlation between two measures represents the portion of shared variance between the measures. Thus, 29% of the .58 personality variance is attributable to a DGF, and the remainder (71%) is due to DSs. As shown in the first data row in Table 1, the DGF variance is estimated to be .17, and the DS variance is .41. The ratio of DGF variance to DS variance is 40.8%, which is in line with Rushton et al.’s (Reference Rushton, Bons, Ando, Hur, Irwing, Vernon and Barbaranelli2009) finding that a DGF accounted for 37% of the personality source variance.
Table 1.Illustration of Consequences of Aggregating Over Multiple Data Sources
We next computed the method variance by calculating the average method effect shown in Table 1 of Podsakoff, MacKenzie, and Podsakoff (Reference Podsakoff, MacKenzie and Podsakoff2012). The average method variance was .25. Podsakoff et al. (Reference Podsakoff, MacKenzie and Podsakoff2012) also reports that the average correlation between methods is typically .47. Using these values, we estimated that the percentage of variance attributable to general (MG) and specific (MS) method variance was .12 and .13, respectively. Finally, the error variance was estimated by subtracting the sum of the aforementioned four systematic variance sources from 1. All of these variance estimates are shown in the first data row of Table 1. As seen in this first row, none of the variance sources is fully dominating the measurement instrument. Kuncel and Sackett (Reference Kuncel and Sackett2014) would say that the DS is moderately dominant in the personality measure.
The subsequent rows in Table 1 indicate what happens as data from multiple sources are combined to yield our overall personality score. The second data row of Table 1 shows the portion of variance attributable to the DGF, DS, MG, MS, and error when information from two different sources (e.g., self-ratings, peer ratings) is combined. The portion of variance not shared by the informant sources (i.e., MS and Error) decreases as aggregation occurs. However, the portion of variance shared across informant sources (i.e., DGF, DS, MG) increases as aggregation occurs. The estimates shown in Table 1 are obtained by using the following equation:
In this equation, k is the number of informant sources being aggregated, and rxF is the correlation between a particular source of variance and a single higher order latent factor. This correlation is estimated by taking the square root of the value in the first data row for that variance source. For example, the estimated correlation between the single latent factor and the portion of the average personality score attributable to a DGF is the square root of .17, or rxF = .410. Equation 1’s numerator when aggregating two informant sources is 2 times .410, or .820.
In the denominator of Equation 1, rxx represents the correlation between the overall personality scores for the two informant samples. This is estima-ted by adding the shared variance components (i.e., σ2DGF + σ2DS+σ2MG), or .70. Equation 1’s denominator is 
$\sqrt {2 + 2( 1 )*.70}$, or approximately 1.84. The DGF entry in the second data row of Table 1 is .820 divided by 1.94, or .44. This estimate, .44, is the correlation between the total personality score aggregated over two informant sources and a single higher order latent factor. Squaring this correlation yields the variance estimate of .198, which rounds to .20 for a DGF. This logic is repeated for all systematic variance source entries. The computation of the noncommon sources of variance is specified in Kuncel and Sackett (Reference Kuncel and Sackett2014).
As shown in Table 1, the systematic sources of variance increase as more informant sources are combined. Table 1 shows that there is a DGF, and its total variance portion increases as more informant sources are combined. However, the DS has a much larger portion of variance, and after aggregating over three informant sources, it is the dominant source in the personality total score. Thus, specific personality dimensions play a bigger role in a measure, provided that information from multiple sources are collected and aggregated.
Do Specific Dimensions Add Value Over DGFs?
The final component of Ree et al.’s argument is that DGFs are really all that matters and the pursuit of DSs is futile. As they note, “analyses that fail to seek DGFs support the myth of the importance of narrow constructs compared with more general constructs” (p. 17). As has been argued elsewhere (e.g., Lievens & Reeve, Reference Lievens and Reeve2012; Reeve, Scherbaum, & Goldstein, Reference Reeve, Scherbaum and Goldstein2015), this position is not helpful to either science or practice that is aimed at understanding the manifestation of individual differences in the workplace. Indeed, this extreme DGF position is not consistent with recent empirical findings that focus on the interplay between general and specific factors. To demonstrate our point, we will switch the literature domain in which the DGF versus DS debate originated: the cognitive ability literature.
In the cognitive ability literature, there is general agreement that the amount of variance accounted for by a DS is small in comparison with the variance accounted for by a general factor (GF; e.g., Gottfredson, Reference Gottfredson1997; Hunter, Reference Hunter1986; McHenry, Hough, Toquam, Hanson, & Ashworth, Reference McHenry, Hough, Toquam, Hanson and Ashworth1990; Ree & Earles, Reference Ree and Earles1991; Ree, Earles, & Teachout, Reference Ree, Earles and Teachout1994; Youngstrom, Kogos, & Glutting, Reference Youngstrom, Kogos and Glutting1999). However, we believe that a critical evaluation of this research and its methodological issues (Reeve, Reference Reeve2004) along with a consideration of all relevant empirical evidence casts doubt on the Ree et al.’s claim that DSs do not matter.
As identified by Reeve (Reference Reeve2004), there are several methodological issues in the previous research that claims that DSs do not matter. One of the primary methodological issues is that the majority of the research examining this question used the observed test battery subscale scores as if they were construct-valid measures of DS constructs (e.g., Hunter, Reference Hunter1986). As Reeve (Reference Reeve2004) argued, the variance in scores from subscales of ability tests that purportedly measure a DS is often confounded because multiple specific and general abilities influence each subscale. In other words, subscales are not necessarily construct-valid assessments of DSs because the DGF has influence in both general and specific measures. When using these measures of DSs to test the relative contribution of DSs and DGFs, one needs to remove the contribution of the DGF (see Gustafsson, Reference Gustafsson, Braun, Jackson and Wiley2002).
Although this point has been recognized previously in some studies, the methods used in these studies are still problematic. For example, Ree and Earles (Reference Ree and Earles1991) attempted to address the problem of confounded variance sources by using principal components to create atheoretical linear components. Ree and Earles did note that their obtained components do not necessarily reflect any specific ability construct, yet they draw conclusions about specific abilities as if they do. Thus, we question how useful the prior research is regarding the relative contribution of the DS and DGF.
Although not acknowledged by Ree et al., there is published research that finds specific factors to be as important as or even more so than a DGF. For example, the research of Lang, Kersting, Hülsheger, and Lang (Reference Lang, Kersting, Hülsheger and Lang2010) finds that when one uses modern analytical techniques (e.g., relative weights analysis) specifically designed to address problems associated with overlapping predictors (i.e., overlap among general and specific ability dimensions as well as overlap among specific abilities dimensions—an issue raised by Ree et al.), the GF accounts for less variance, and the DSs are more useful than previously believed. Similarly, Wee, Newman, and Joseph (Reference Wee, Newman and Joseph2014) find DSs are valuable when modern analytical techniques are used. Studies such as Lang et al. and Wee et al. raise questions about the viability of extreme positions such as the claim that that only DGFs matter. These claims yield a false impression that this issue has been conclusively answered.
Moreover, the new cognitive ability literature raises questions about the appropriateness of focusing on competitive tests comparing GF and DSs. For example, Lubinski and colleagues’ (e.g., Park, Lubinski, & Benbow, Reference Park, Lubinski and Benbow2008; Wai, Reference Wai2013) research examining the role of specific abilities among high ability groups suggests that GF and DSs work together and that both are necessary to understand intellectual behavior. In their longitudinal study of gifted adolescents, Park et al. analyzed the role that ability level and ability tilt play in professional accomplishments over a 25-year time span. Ability tilt refers to an asymmetry in DSs across different domains. Park et al. examined ability tilt between math and verbal ability. Larger differences between math and verbal SAT scores are indicative of larger ability tilts. Park et al. found that the ability tilts identified at age 13 foreshadowed contrasting forms of professional accomplishment in middle age. Specifically, they showed that although GF contributed to accomplishments, ability tilt was critical for predicting the domain in which these achievements occurred (e.g., securing a tenure-track position in the humanities vs. STEM sciences; publishing a novel vs. securing a patent) over 25 years later. Lubinski and colleagues found similar results for spatial ability (Shea, Lubinski, & Benbow, Reference Shea, Lubinski and Benbow2001; Webb, Lubinski, & Benbow, Reference Webb, Lubinski and Benbow2007).
Theoretical and empirical work similar to Lubinski and colleagues strongly suggests that research seeking to paint a dichotomy between the GF and DS is asking the wrong question. Recent cognitive ability theoretic models emphasize the interplay between general and specific abilities both within and between domains. Essentially, this new theoretical work emphasizes the constellation of interacting DSs that combine to explain individual differences in behaviors. Snow's comprehensive theory of aptitude (Corno et al., Reference Corno, Cronbach, Kupermintz, Lohman, Mandinach and Talbert2002; Snow, Reference Snow1987, Reference Snow1992), Ackerman's (Reference Ackerman1996) intelligence-as-process, personality, interests, and intelligence-as-knowledge (PPIK) theory, and Chamorro-Premuzic and Furnham's (Reference Chamorro-Premuzic and Furnham2005) emerging model of intellectual competence are exemplars of modern thinking emphasizing the collaborative nature (as opposed to competitive nature) of general and specific abilities. Even modern psychometric models of intelligence (e.g., Cattell-Horn-Carroll model of intelligence) emphasize the importance of both DGFs and DSs (Schneider & McGrew, Reference Schneider, McGrew, Flanagan and Harrison2012; Schneider & Newman, Reference Schneider and Newman2015). Finally, even though Ree et al. dismiss the work of van der Maas et al. (Reference van der Maas, Dolan, Grasman, Wicherts, Huizenga and Raijmakers2006), these researchers come to similar conclusions that specific factors are important and dichotomous questions contrasting the GF with the DS are simply not helpful for understanding human behavior and development.
Conclusions
We provided counterarguments against three major points raised in Ree et al.’s article. First, using the personality literature as an example case, we showed that the presence of a DGF is still actively debated and that the empirical evidence supportive of a DGF is not as clean as implied by Ree et al. Second, using psychometric theory, we illustrated how the influence of a DS can grow until it is the dominant source of variance in a measure when multiple sources of data about a target individual are combined. This illustration is consistent with DeYoung's published findings demonstrating that support for a DGF is eliminated when data from multiple informant sources are combined in the statistical model used to test for a DGF. Finally, although debates about the relative value of broad versus specific factors are common in the organizational sciences and can be helpful for scientific progress (Judge & Kammeyer-Mueller, Reference Judge and Kammeyer-Mueller2012), we argue, as have others (Hogan & Roberts, Reference Hogan and Roberts1996; Judge & Kammeyer-Mueller, Reference Judge and Kammeyer-Mueller2012; Reeve & Hakel, Reference Reeve and Hakel2002), that extreme positions such as those expressed by Ree et al. are not scientifically justified, nor are they helpful for promoting science and practice. The false all-or-nothing dichotomies expressed in these debates are not productive and are asking the wrong questions. Given that DSs are the focus of many workplace applications and processes, the extreme stance against DSs puts us further out of touch of the pressing needs of practitioners and may not be helpful for increasing our understanding of the manifestations of individual differences in the workplace.
%20-%20https://sketch.com%20--%3e%3ctitle%3e3ECBBF50-FD08-412A-943D-2CFC5E025A16%3c/title%3e%3cdesc%3eCreated%20with%20sketchtool.%3c/desc%3e%3cg%20id='Stylesheet'%20stroke='none'%20stroke-width='1'%20fill='none'%20fill-rule='evenodd'%3e%3cg%20transform='translate(-926.000000,%20-668.000000)'%20id='BUTTONS'%3e%3cg%20transform='translate(100.000000,%20597.000000)'%3e%3cg%20id='BTN-/-RIGHTS'%20transform='translate(816.000000,%2065.000000)'%3e%3cg%20id='ICON-/-Rights'%20transform='translate(10.000000,%206.000000)'%3e%3cpath%20d='M13.5752841,0%20L1.34943182,0%20C0.59375,0%200,0.586666667%200,1.33333333%20L0,22.6666667%20C0,23.4133333%200.620738636,24%201.34943182,24%20L17.6505682,24%20C18.40625,24%2019,23.3866667%2019,22.6666667%20L19,5.36%20L13.5752841,0%20Z%20M13.5752841,1.89333333%20L17.0838068,5.36%20L13.5752841,5.36%20L13.5752841,1.89333333%20Z%20M1.34943182,22.6666667%20L1.34943182,1.33333333%20L12.1988636,1.33333333%20L12.1988636,5.36%20C12.1988636,6.10666667%2012.7926136,6.69333333%2013.5482955,6.69333333%20L17.6235795,6.69333333%20L17.6505682,22.6666667%20L1.34943182,22.6666667%20Z'%20id='Shape'%20fill='%23595959'%20fill-rule='nonzero'%3e%3c/path%3e%3ccircle%20id='Oval'%20stroke='%23595959'%20stroke-width='1.5'%20fill='%23595959'%20cx='9.5'%20cy='13.5355339'%20r='5.5'%3e%3c/circle%3e%3cpolyline%20id='Path-2'%20stroke='%23FFFFFF'%20stroke-width='1.5'%20transform='translate(9.500000,%2013.035534)%20rotate(45.000000)%20translate(-9.500000,%20-13.035534)%20'%20points='8%2016.0355339%2011%2016.0355339%2011%2010.0355339'%3e%3c/polyline%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
In their article, Ree, Carretta, and Teachout (Reference Ree, Carretta and Teachout2015) argued that a dominant general factor (DGF) is present in most, if not all, psychological measures (e.g., personality, leadership, attitudes, skills). A DGF, according to Ree et al., is identified by two characteristics. First, the DGF accounts for the largest amount of a measure's systematic variance, and second, it influences every subdimension within the construct domain. They indicate that researchers ignore DGFs and pay inappropriate amounts of attention to the specific dimensions (DSs) even though the DGF provides most of the predictive power and the DS adds little predictive power.
In this article, we raise three issues with Ree et al.’s arguments. First, we argue that the evidence for the DGF, at least in one content area (i.e., personality), is not clean cut. Second, we will show that even if there is a DGF in personality measures, the contribution of specific personality dimensions becomes the more dominant source in the measure once monomethod/mono-operational biases are accounted for in the data. Third, we argue that, in contrast to Ree et al., there is growing evidence that DSs in the very domain that originated the DGF debate, cognitive ability, are useful and have predictive power.
Universal DGF? The Case of Personality
Is there really a personality DGF? Although there are publications supporting this claim, this idea is still actively debated (e.g., Ashton, Lee, Goldberg & De Vries, Reference Ashton, Lee, Goldberg and de Vries2009; Hopwood, Wright, & Donnellan, Reference Hopwood, Wright and Donnellan2011), and there is counterevidence against a personality DGF. It is important to recognize these counterarguments to prevent a false impression that a DGF in the personality literature is widely accepted.
Examples of counterevidence to a DGF include Donnellan, Hopwood, and Wright's (Reference Donnellan, Hopwood and Wright2012) attempt to replicate the Rushton and Irwing (Reference Rushton and Irwing2008) study that supported a personality DGF. Using a new sample, Donnellan et al. (Reference Donnellan, Hopwood and Wright2012) failed to find support for a DGF model. Although failing to replicate findings may be a sign of our times, it is more disturbing that Donnellan et al. couldn't replicate the published DGF model when they recreated Rushton and Irwing's variance/covariance matrix using the published correlation matrix combined with variable standard deviations. Donnellan et al. report that they could recapture the DGF model's published degrees of freedom only by adding additional constraints not reported in the original article. Surprisingly, even with these constraints, the DGF model would not converge with Rushton and Irwing variance/covariance matrix nor with their own sample.
Rather than a DGF, many studies found two higher order factors (e.g., DeYoung, Peterson, & Higgins, Reference DeYoung, Peterson and Higgins2002; Digman, Reference Digman1997). Agreeableness, Conscientiousness, and Emotional Stability loaded on one higher order factor called the alpha or stability metatrait factor. Extraversion and Openness to Experience loaded on another higher order factor called the beta or plasticity metatrait factor (DeYoung et al., Reference DeYoung, Peterson and Higgins2002; Digman, Reference Digman1997). Using Ree et al.’s definitions, these metatrait factors are group factors, not DGFs, because each metatrait factor influences only some of the personality dimensions. The question now appears to be not whether the Big Five load onto a single DGF but whether the two aforementioned group factors are sufficiently correlated to justify a third level DGF. The jury is still out because the correlations between the two group factors have ranged from being uncorrelated (DeYoung, Reference DeYoung2006) to being values between .18 and .48 (DeYoung, Peterson, & Higgins, Reference DeYoung, Peterson and Higgins2002; Musek, Reference Musek2007). The smaller the correlation between the two group factors, the less likely it is that a DGF exists.
DeYoung (Reference DeYoung2006) found that smaller group factor correlations are obtained when the variance associated with different types of data sources (e.g., self-ratings, spouses, friends) is incorporated in the statistical analyses. Specifically, higher correlations between the alpha and beta group factors were obtained when he performed statistical analyses separately for each data source (i.e., analysis done for only self-ratings; another analysis performed using friend data). Interestingly, the two group factors were uncorrelated when he reanalyzed the data and incorporated all data sources explicitly in his analyses.
In summary, declaring that there is a DGF among personality dimensions is premature. This may also be true with the other domains covered in Ree et al.
Dominance of Specific Dimensions in Measures
Even if there is a personality DGF, the DSs can still be the dominant source of influence in personality measures. To illustrate how this could be true, we followed the psychometric and mathematical logic outlined by Kuncel and Sackett (Reference Kuncel and Sackett2014) when they discussed assessment center ratings. First, let us assume that we have a Big Five instrument and that we want to create a single personality score for each individual by adding the dimension score ratings together. This would be reasonable to do if one believed in a personality DGF.
As with any measurement instrument, there are three sources of variance that affect personality scores: (a) personality itself, (b) method bias, and (c) random error. The first two variance sources are systematic and can be split into a general and a specific variance portion. This means that we are assuming there is a general personality (i.e., DGF) variance source and a source of variance attributable to specific personality dimensions (DSs). Similarly, we assume that there is a general method variance portion (method general; MG) due to similar characteristics across all methods and a method specific (MS) variance portion that is unique to each method.
Following the findings of Rushton, Bons, and Hur (Reference Rushton, Bons and Hur2008), we set the percentage of variance attributable to personality (both general and specific) to be 58%. We used Mount, Barrick, Scullen, and Rounds's (Reference Mount, Barrick, Scullen and Rounds2005) reliability-corrected correlations among the Big Five dimensions to estimate the average Big Five correlation to be .29. According to psychometric theory (Ghiselli, Campbell, & Zedeck, Reference Ghiselli, Campbell and Zedeck1981), the correlation between two measures represents the portion of shared variance between the measures. Thus, 29% of the .58 personality variance is attributable to a DGF, and the remainder (71%) is due to DSs. As shown in the first data row in Table 1, the DGF variance is estimated to be .17, and the DS variance is .41. The ratio of DGF variance to DS variance is 40.8%, which is in line with Rushton et al.’s (Reference Rushton, Bons, Ando, Hur, Irwing, Vernon and Barbaranelli2009) finding that a DGF accounted for 37% of the personality source variance.
Illustration of Consequences of Aggregating Over Multiple Data Sources
Note. DGF = dominant general factor; DS = specific dimension; MG = method general; MS = method specific.
We next computed the method variance by calculating the average method effect shown in Table 1 of Podsakoff, MacKenzie, and Podsakoff (Reference Podsakoff, MacKenzie and Podsakoff2012). The average method variance was .25. Podsakoff et al. (Reference Podsakoff, MacKenzie and Podsakoff2012) also reports that the average correlation between methods is typically .47. Using these values, we estimated that the percentage of variance attributable to general (MG) and specific (MS) method variance was .12 and .13, respectively. Finally, the error variance was estimated by subtracting the sum of the aforementioned four systematic variance sources from 1. All of these variance estimates are shown in the first data row of Table 1. As seen in this first row, none of the variance sources is fully dominating the measurement instrument. Kuncel and Sackett (Reference Kuncel and Sackett2014) would say that the DS is moderately dominant in the personality measure.
The subsequent rows in Table 1 indicate what happens as data from multiple sources are combined to yield our overall personality score. The second data row of Table 1 shows the portion of variance attributable to the DGF, DS, MG, MS, and error when information from two different sources (e.g., self-ratings, peer ratings) is combined. The portion of variance not shared by the informant sources (i.e., MS and Error) decreases as aggregation occurs. However, the portion of variance shared across informant sources (i.e., DGF, DS, MG) increases as aggregation occurs. The estimates shown in Table 1 are obtained by using the following equation:
In this equation, k is the number of informant sources being aggregated, and rxF is the correlation between a particular source of variance and a single higher order latent factor. This correlation is estimated by taking the square root of the value in the first data row for that variance source. For example, the estimated correlation between the single latent factor and the portion of the average personality score attributable to a DGF is the square root of .17, or rxF = .410. Equation 1’s numerator when aggregating two informant sources is 2 times .410, or .820.
In the denominator of Equation 1, rxx represents the correlation between the overall personality scores for the two informant samples. This is estima-ted by adding the shared variance components (i.e., σ2DGF + σ2DS+σ2MG), or .70. Equation 1’s denominator is
$\sqrt {2 + 2( 1 )*.70}$, or approximately 1.84. The DGF entry in the second data row of Table 1 is .820 divided by 1.94, or .44. This estimate, .44, is the correlation between the total personality score aggregated over two informant sources and a single higher order latent factor. Squaring this correlation yields the variance estimate of .198, which rounds to .20 for a DGF. This logic is repeated for all systematic variance source entries. The computation of the noncommon sources of variance is specified in Kuncel and Sackett (Reference Kuncel and Sackett2014).
As shown in Table 1, the systematic sources of variance increase as more informant sources are combined. Table 1 shows that there is a DGF, and its total variance portion increases as more informant sources are combined. However, the DS has a much larger portion of variance, and after aggregating over three informant sources, it is the dominant source in the personality total score. Thus, specific personality dimensions play a bigger role in a measure, provided that information from multiple sources are collected and aggregated.
Do Specific Dimensions Add Value Over DGFs?
The final component of Ree et al.’s argument is that DGFs are really all that matters and the pursuit of DSs is futile. As they note, “analyses that fail to seek DGFs support the myth of the importance of narrow constructs compared with more general constructs” (p. 17). As has been argued elsewhere (e.g., Lievens & Reeve, Reference Lievens and Reeve2012; Reeve, Scherbaum, & Goldstein, Reference Reeve, Scherbaum and Goldstein2015), this position is not helpful to either science or practice that is aimed at understanding the manifestation of individual differences in the workplace. Indeed, this extreme DGF position is not consistent with recent empirical findings that focus on the interplay between general and specific factors. To demonstrate our point, we will switch the literature domain in which the DGF versus DS debate originated: the cognitive ability literature.
In the cognitive ability literature, there is general agreement that the amount of variance accounted for by a DS is small in comparison with the variance accounted for by a general factor (GF; e.g., Gottfredson, Reference Gottfredson1997; Hunter, Reference Hunter1986; McHenry, Hough, Toquam, Hanson, & Ashworth, Reference McHenry, Hough, Toquam, Hanson and Ashworth1990; Ree & Earles, Reference Ree and Earles1991; Ree, Earles, & Teachout, Reference Ree, Earles and Teachout1994; Youngstrom, Kogos, & Glutting, Reference Youngstrom, Kogos and Glutting1999). However, we believe that a critical evaluation of this research and its methodological issues (Reeve, Reference Reeve2004) along with a consideration of all relevant empirical evidence casts doubt on the Ree et al.’s claim that DSs do not matter.
As identified by Reeve (Reference Reeve2004), there are several methodological issues in the previous research that claims that DSs do not matter. One of the primary methodological issues is that the majority of the research examining this question used the observed test battery subscale scores as if they were construct-valid measures of DS constructs (e.g., Hunter, Reference Hunter1986). As Reeve (Reference Reeve2004) argued, the variance in scores from subscales of ability tests that purportedly measure a DS is often confounded because multiple specific and general abilities influence each subscale. In other words, subscales are not necessarily construct-valid assessments of DSs because the DGF has influence in both general and specific measures. When using these measures of DSs to test the relative contribution of DSs and DGFs, one needs to remove the contribution of the DGF (see Gustafsson, Reference Gustafsson, Braun, Jackson and Wiley2002).
Although this point has been recognized previously in some studies, the methods used in these studies are still problematic. For example, Ree and Earles (Reference Ree and Earles1991) attempted to address the problem of confounded variance sources by using principal components to create atheoretical linear components. Ree and Earles did note that their obtained components do not necessarily reflect any specific ability construct, yet they draw conclusions about specific abilities as if they do. Thus, we question how useful the prior research is regarding the relative contribution of the DS and DGF.
Although not acknowledged by Ree et al., there is published research that finds specific factors to be as important as or even more so than a DGF. For example, the research of Lang, Kersting, Hülsheger, and Lang (Reference Lang, Kersting, Hülsheger and Lang2010) finds that when one uses modern analytical techniques (e.g., relative weights analysis) specifically designed to address problems associated with overlapping predictors (i.e., overlap among general and specific ability dimensions as well as overlap among specific abilities dimensions—an issue raised by Ree et al.), the GF accounts for less variance, and the DSs are more useful than previously believed. Similarly, Wee, Newman, and Joseph (Reference Wee, Newman and Joseph2014) find DSs are valuable when modern analytical techniques are used. Studies such as Lang et al. and Wee et al. raise questions about the viability of extreme positions such as the claim that that only DGFs matter. These claims yield a false impression that this issue has been conclusively answered.
Moreover, the new cognitive ability literature raises questions about the appropriateness of focusing on competitive tests comparing GF and DSs. For example, Lubinski and colleagues’ (e.g., Park, Lubinski, & Benbow, Reference Park, Lubinski and Benbow2008; Wai, Reference Wai2013) research examining the role of specific abilities among high ability groups suggests that GF and DSs work together and that both are necessary to understand intellectual behavior. In their longitudinal study of gifted adolescents, Park et al. analyzed the role that ability level and ability tilt play in professional accomplishments over a 25-year time span. Ability tilt refers to an asymmetry in DSs across different domains. Park et al. examined ability tilt between math and verbal ability. Larger differences between math and verbal SAT scores are indicative of larger ability tilts. Park et al. found that the ability tilts identified at age 13 foreshadowed contrasting forms of professional accomplishment in middle age. Specifically, they showed that although GF contributed to accomplishments, ability tilt was critical for predicting the domain in which these achievements occurred (e.g., securing a tenure-track position in the humanities vs. STEM sciences; publishing a novel vs. securing a patent) over 25 years later. Lubinski and colleagues found similar results for spatial ability (Shea, Lubinski, & Benbow, Reference Shea, Lubinski and Benbow2001; Webb, Lubinski, & Benbow, Reference Webb, Lubinski and Benbow2007).
Theoretical and empirical work similar to Lubinski and colleagues strongly suggests that research seeking to paint a dichotomy between the GF and DS is asking the wrong question. Recent cognitive ability theoretic models emphasize the interplay between general and specific abilities both within and between domains. Essentially, this new theoretical work emphasizes the constellation of interacting DSs that combine to explain individual differences in behaviors. Snow's comprehensive theory of aptitude (Corno et al., Reference Corno, Cronbach, Kupermintz, Lohman, Mandinach and Talbert2002; Snow, Reference Snow1987, Reference Snow1992), Ackerman's (Reference Ackerman1996) intelligence-as-process, personality, interests, and intelligence-as-knowledge (PPIK) theory, and Chamorro-Premuzic and Furnham's (Reference Chamorro-Premuzic and Furnham2005) emerging model of intellectual competence are exemplars of modern thinking emphasizing the collaborative nature (as opposed to competitive nature) of general and specific abilities. Even modern psychometric models of intelligence (e.g., Cattell-Horn-Carroll model of intelligence) emphasize the importance of both DGFs and DSs (Schneider & McGrew, Reference Schneider, McGrew, Flanagan and Harrison2012; Schneider & Newman, Reference Schneider and Newman2015). Finally, even though Ree et al. dismiss the work of van der Maas et al. (Reference van der Maas, Dolan, Grasman, Wicherts, Huizenga and Raijmakers2006), these researchers come to similar conclusions that specific factors are important and dichotomous questions contrasting the GF with the DS are simply not helpful for understanding human behavior and development.
Conclusions
We provided counterarguments against three major points raised in Ree et al.’s article. First, using the personality literature as an example case, we showed that the presence of a DGF is still actively debated and that the empirical evidence supportive of a DGF is not as clean as implied by Ree et al. Second, using psychometric theory, we illustrated how the influence of a DS can grow until it is the dominant source of variance in a measure when multiple sources of data about a target individual are combined. This illustration is consistent with DeYoung's published findings demonstrating that support for a DGF is eliminated when data from multiple informant sources are combined in the statistical model used to test for a DGF. Finally, although debates about the relative value of broad versus specific factors are common in the organizational sciences and can be helpful for scientific progress (Judge & Kammeyer-Mueller, Reference Judge and Kammeyer-Mueller2012), we argue, as have others (Hogan & Roberts, Reference Hogan and Roberts1996; Judge & Kammeyer-Mueller, Reference Judge and Kammeyer-Mueller2012; Reeve & Hakel, Reference Reeve and Hakel2002), that extreme positions such as those expressed by Ree et al. are not scientifically justified, nor are they helpful for promoting science and practice. The false all-or-nothing dichotomies expressed in these debates are not productive and are asking the wrong questions. Given that DSs are the focus of many workplace applications and processes, the extreme stance against DSs puts us further out of touch of the pressing needs of practitioners and may not be helpful for increasing our understanding of the manifestations of individual differences in the workplace.