IQ tests vs. assessment centers:
What should be chosen for valid predictions?
Blog on the article by Sackett, P. R., Shewach, O. R., & Keiser, H. N.. (2017). Assessment centers versus cognitive ability tests: Challenging the conventional wisdom on criterion-related validity. Journal of Applied Psychology. Advance Online Publication. doi:10.1037/apl0000236.
Superiority of intelligence tests has long times suggested by previous meta-analyses (e.g., Schmidt & Hunter, 1998), so this knowledge has also long been considered established in the practice and textbook literature. Sackett and colleagues addressed recently again the question of whether candidate job performance is better predicted by IQ scores than by assessment center performance, or in other words, whether intelligence tests are preferable to assessment centers.
Unlike previous meta-analyses, however, they only included studies (3 800 individuals in total) in their comparison in which participants completed both IQ tests and assessment centers. This ensured that the range of assessment center results was not limited by preselection with respect to intelligence like in other studies. They found that the (corrected) criterion validity (= the relationship to job performance as a measure of quality for the selection process) of the assessment center for prediction was on average .44, while it was only .22 for the intelligence test. Compared to Schmidt and Hunter (1998) the coefficients were contrary, .37 vs. 58.
Additionally, Sackett and colleagues reported that in 15 of 17 data sets the criterion validity of the assessment center was higher than the criterion validity of the intelligence test. This meta-analysis upends long-established knowledge and demonstrates that assessment centers have been wrongly judged inferior to intelligence tests in predicting job performance. On the contrary, the findings of this meta-analysis actually suggest that assessment centers may be superior to intelligence tests in predicting job performance. The main reasons for these different results may be range restriction and different, cognitive oriented criteria in the older studies.
Overall, the findings of this study are definitely a reason for optimism for assessment center enthusiasts, as in principle a gain from the additional use of assessment centers over the significantly less resource-intensive IQ tests seems plausible. However, validation in the organization still remains central, since the criterion validity of the assessment center - in contrast to highly standardized IQ tests - depends on the design and execution chosen in each case.
Written by Martin Kleinmann
m.kleinmann@psychologie.uzh.ch
30 November 2021
Superiority of intelligence tests has long times suggested by previous meta-analyses (e.g., Schmidt & Hunter, 1998), so this knowledge has also long been considered established in the practice and textbook literature. Sackett and colleagues addressed recently again the question of whether candidate job performance is better predicted by IQ scores than by assessment center performance, or in other words, whether intelligence tests are preferable to assessment centers.
Unlike previous meta-analyses, however, they only included studies (3 800 individuals in total) in their comparison in which participants completed both IQ tests and assessment centers. This ensured that the range of assessment center results was not limited by preselection with respect to intelligence like in other studies. They found that the (corrected) criterion validity (= the relationship to job performance as a measure of quality for the selection process) of the assessment center for prediction was on average .44, while it was only .22 for the intelligence test. Compared to Schmidt and Hunter (1998) the coefficients were contrary, .37 vs. 58.
Additionally, Sackett and colleagues reported that in 15 of 17 data sets the criterion validity of the assessment center was higher than the criterion validity of the intelligence test. This meta-analysis upends long-established knowledge and demonstrates that assessment centers have been wrongly judged inferior to intelligence tests in predicting job performance. On the contrary, the findings of this meta-analysis actually suggest that assessment centers may be superior to intelligence tests in predicting job performance. The main reasons for these different results may be range restriction and different, cognitive oriented criteria in the older studies.
Overall, the findings of this study are definitely a reason for optimism for assessment center enthusiasts, as in principle a gain from the additional use of assessment centers over the significantly less resource-intensive IQ tests seems plausible. However, validation in the organization still remains central, since the criterion validity of the assessment center - in contrast to highly standardized IQ tests - depends on the design and execution chosen in each case.
Written by Martin Kleinmann
m.kleinmann@psychologie.uzh.ch
30 November 2021