Hostname: page-component-7b7567c8c-wxvwp Total loading time: 0 Render date: 2026-11-18T14:57:28.320Z Has data issue: false hasContentIssue false

Corrections for Criterion Reliability in Validity Generalization: A False Prophet in a Land of Suspended Judgment

Published online by Cambridge University Press:  10 April 2015

James M. LeBreton*
Affiliation:
Pennsylvania State University
Kelly T. Scherer
Affiliation:
Purdue University
Lawrence R. James
Affiliation:
Georgia Institute of Technology
*
E-mail: james.lebreton@psu.edu, Address: Department of Psychology, Pennsylvania State University, 140 Moore Building, University Park, PA 16802
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

The results of meta-analytic (MA) and validity generalization (VG) studies continue to be impressive. In contrast to earlier findings that capped the variance accounted for in job performance at roughly 16%, many recent studies suggest that a single predictor variable can account for between 16 and 36% of the variance in some aspect of job performance. This article argues that this “enhancement” in variance accounted for is often attributable not to improvements in science but to a dumbing down of the standards for the values of statistics used in correction equations. With rare exceptions, applied researchers have suspended judgment about what is and is not an acceptable threshold for criterion reliability in their quest for higher validities. We demonstrate a statistical dysfunction that is a direct result of using low criterion reliabilities in corrections for attenuation. Corrections typically applied to a single predictor in a VG study are instead applied to multiple predictors. A multiple correlation analysis is then conducted on corrected validity coefficients. It is shown that the corrections often used in single predictor studies yield a squared multiple correlation that appears suspect. Basically, the multiple predictor study exposes the tenuous statistical foundation of using abjectly low criterion reliabilities in single predictor VG studies. Recommendations for restoring scientific integrity to the meta-analyses that permeate industrial–organizational (I–O) psychology are offered.

Information

Type
Focal Article
Copyright
Copyright © Society for Industrial and Organizational Psychology 2014