Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Original Article

Examining the Incremental Validity and Relative Importance of Specific Cognitive Abilities in a Training Context

Published Online:August 05, 2014https://doi.org/10.1027/1866-5888/a000116

Abstract

We examine the extent to which g and specific abilities predict learning in a training context by juxtaposing contrasting theories of cognitive ability (i.e., Spearmanian theory and nested-factors theory) and using an analytical strategy (i.e., multidimensional importance analysis) that aligns with underlying theoretical assumptions of both theories. When conducting incremental validity analysis (Spearmanian approach), specific abilities did not add much to the prediction of learning. However, when conducting relative weights analysis (nested-factors approach), specific abilities were the dominant predictors of learning. Results suggest different theoretical and analytical approaches can lead to different results. Results also suggest specific abilities may provide predictive utility beyond g in the training context, but recognizing this utility depends on one’s theoretical assumptions and methodological approach.

References

  • Ackerman, P. L. (1987). Individual differences in skill learning: An integration of psychometric and information processing perspectives. Psychological Bulletin, 102, 3–27. doi: 10.1037/0033-2909.102.1.3 First citation in articleCrossref, Google Scholar

  • Anderson, J. C. , Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411–423. doi: 10.1037/0033-2909.103.3.411 First citation in articleCrossref, Google Scholar

  • Austin, J. T. , Villanova, P. (1992). The criterion problem: 1917–1992. Journal of Applied Psychology, 77, 836–874. doi: 10.1037/0021-9010.77.6.836 First citation in articleCrossref, Google Scholar

  • Brogden, H. E. (1946). On the interpretation of the correlation coefficient as a measure of predictive efficiency. Journal of Educational Psychology, 37, 65–76. First citation in articleCrossref, Google Scholar

  • Brogden, H. E. (1949). When testing pays off. Personnel Psychology, 2, 171–183. First citation in articleCrossref, Google Scholar

  • Brown, K. G. , Le, H. , Schmidt, F. L. (2006). Specific aptitude theory revisited: Is there incremental validity for training performance? International Journal of Selection and Assessment, 14, 87–100. doi: 10.1111/j.1468-2389.2006.00336.x First citation in articleCrossref, Google Scholar

  • Budescu, D. V. (1993). Dominance analysis: A new approach to the problem of relative importance of predictors in multiple regression. Psychological Bulletin, 114, 542–551. doi: 10.1037/0033-2909.114.3.542 First citation in articleCrossref, Google Scholar

  • Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York, NY: Cambridge University Press. First citation in articleCrossref, Google Scholar

  • Chen, G. , Gully, S. M. , Whiteman, J. , Kilcullen, R. N. (2000). Examination of relationships among trait-like individual differences, state-like individual differences, and learning performance. Journal of Applied Psychology, 85, 835–847. doi: 10.1037/0021-9010.85.6.835 First citation in articleCrossref, Google Scholar

  • Cohen, J. , & Cohen, P. (1983). Applied multiple regression/correlation analyses for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Cronbach, L. J. , Gleser, G. C. (1957). Psychological tests and personnel decisions. Champaign, IL: University of Illinois Press. First citation in articleGoogle Scholar

  • Cronbach, L. J. , Gleser, G. C. (1965). Psychological tests and personnel decisions. Oxford, UK: University of Illinois Press. First citation in articleGoogle Scholar

  • Darlington, R. B. (1968). Multiple regression in psychological research and practice. Psychological Bulletin, 69, 161–182. doi: 10.1037/h0025471 First citation in articleCrossref, Google Scholar

  • DeShon, R. P. (1998). A cautionary note on measurement error corrections in structural equation models. Psychological Methods, 3, 412–423. doi: 10.1037/1082-989X.3.4.412 First citation in articleCrossref, Google Scholar

  • Dierdorff, E. C. , Surface, E. A. (2008). If you pay for skills, will they learn? Skill change and maintenance under a skill-based pay system. Journal of Management, 34, 721–743. doi: 10.1177/0149206307312507 First citation in articleCrossref, Google Scholar

  • Dubin, R. (1976). Theory building in applied areas. In M. D. Dunnette, (Ed.), Handbook of industrial and organizational psychology (pp. 17–39). Chicago, IL: Rand McNally. First citation in articleGoogle Scholar

  • Goldstein, I. L. , Ford, J. K. (2002). Training in organizations: Needs assessment, development, and evaluation (4th ed.). Belmont, CA, US: Wadsworth/Thomson Learning. First citation in articleGoogle Scholar

  • Gottfredson, L. S. (1997). Why g matters: The complexity of everyday life. Intelligence, 24, 79–132. doi: 10.1016/S0160-2896(97)90014-3 First citation in articleCrossref, Google Scholar

  • Green, M. , & McGill, E. (2011). State of the industry, 2011: ASTD’s annual review of workplace learning and development data. Alexandria, VA: ASTD. First citation in articleGoogle Scholar

  • Gustafsson, J. , Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28, 407–434. doi: 10.1207/s15327906mbr2804_2 First citation in articleCrossref, Google Scholar

  • Hogan, J. , & Roberts, B. W. (1996). Issues and non-issues in the fidelity-bandwidth trade-off. Journal of Organizational Behavior, 17, 627–637. doi: 10.1002/(SICI)1099-1379(199611)17:6<627::AID-JOB2828>3.0.CO;2-F First citation in articleCrossref, Google Scholar

  • Hunter, J. E. (1986). Cognitive ability, cognitive aptitude, job knowledge, and job performance. Journal of Vocational Behavior, 29, 340–362. doi: 10.1016/0001-8791(86)90013-8 First citation in articleCrossref, Google Scholar

  • Hunter, J. E. , Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72–98. doi: 10.1037/0033-2909.96.1.72 First citation in articleCrossref, Google Scholar

  • Johnson, J. W. (2000). A heuristic method for estimating the relative weight of predictor variables in multiple regression. Multivariate Behavioral Research, 35, 1–19. doi: 10.1207/S15327906MBR3501_1 First citation in articleCrossref, Google Scholar

  • Johnson, J. W. , LeBreton, J. M. (2004). History and use of relative importance indices in organizational research. Organizational Research Methods, 7, 238–257. doi: 10.1177/1094428104266510 First citation in articleCrossref, Google Scholar

  • Kass, R. A. , Mitchell, K. J. , Grafton, F. C. , Wing, H. (1983). Factorial validity of the armed services vocational aptitude battery (ASVAB), Forms 8, 9 and 10: 1981 Army applicant sample. Educational and Psychological Measurement, 43, 1077–1087. doi: 10.1177/001316448304300417 First citation in articleCrossref, Google Scholar

  • Kraiger, K. , Ford, J. , Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, 311–328. doi: 10.1037/0021-9010.78.2.311 First citation in articleCrossref, Google Scholar

  • Lang, J. B. , Bliese, P. D. (2012). I–O psychology and progressive research programs on intelligence. Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 161–166. doi: 10.1111/j.1754-9434.2012.01423.x First citation in articleCrossref, Google Scholar

  • Lang, J. B. , Kersting, M. , Hülsheger, U. R. , Lang, J. (2010). General mental ability, narrower cognitive abilities, and job performance: The perspective of the nested-factors model of cognitive abilities. Personnel Psychology, 63, 595–640. doi: 10.1111/j.1744-6570.2010.01182.x First citation in articleCrossref, Google Scholar

  • LeBreton, J. M. , Hargis, M. B. , Griepentrog, B. , Oswald, F. L. , Ployhart, R. E. (2007). A multidimensional approach for evaluating variables in organizational research and practice. Personnel Psychology, 60, 475–498. doi: 10.1111/j.1744-6570.2007.00080.x First citation in articleCrossref, Google Scholar

  • LeBreton, J. M. , Ployhart, R. E. , Ladd, R. T. (2004). A Monte Carlo comparison of relative importance methodologies. Organizational Research Methods, 7, 258–282. doi: 10.1177/1094428104266017 First citation in articleCrossref, Google Scholar

  • Mount, M. K. , Oh, I. , Burns, M. (2008). Incremental validity of perceptual speed and accuracy over general mental ability. Personnel Psychology, 61, 113–139. doi: 10.1111/j.1744-6570.2008.00107.x First citation in articleCrossref, Google Scholar

  • Noe, R. A. (2005). Employee training and development (3rd ed.). Burr Ridge, IL: Irwin McGraw-Hill. First citation in articleGoogle Scholar

  • Petersen, C. R. , Al-Haik, A. R. (1976). The development of the defense language aptitude battery (DLAB). Educational and Psychological Measurement, 36, 369–380. doi: 10.1177/001316447603600216 First citation in articleCrossref, Google Scholar

  • Ree, M. J. , & Earles, J. A. (1991a). Predicting training success: Not much more than g . Personnel Psychology, 44, 321–332. doi: 10.1111/j.1744-6570.1991.tb00961.x First citation in articleCrossref, Google Scholar

  • Ree, M. J. , & Earles, J. A. (1991b). The stability of g across different methods of estimation. Intelligence, 15, 271–278. doi: 10.1016/0160-2896(91)90036-D First citation in articleCrossref, Google Scholar

  • Ree, M. , Earles, J. A. , Teachout, M. S. (1994). Predicting job performance: Not much more than g . Journal of Applied Psychology, 79, 518–524. doi: 10.1037/0021-9010.79.4.518 First citation in articleCrossref, Google Scholar

  • Salgado, J. F. , Anderson, N. , Moscoso, S. , Bertua, C. , de Fruyt, F. , Rolland, J. (2003). A meta-analytic study of general mental ability validity for different occupations in the European Community. Journal of Applied Psychology, 88, 1068–1081. doi: 10.1037/0021-9010.88.6.1068 First citation in articleCrossref, Google Scholar

  • Schmidt, F. L. , Hunter, J. (2004). General mental ability in the world of work: Occupational attainment and job performance. Journal of Personality and Social Psychology, 86, 162–173. doi: 10.1037/0022-3514.86.1.162 First citation in articleCrossref, Google Scholar

  • Schmidt, F. L. , Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274. doi: 10.1037/0033-2909.124.2.262 First citation in articleCrossref, Google Scholar

  • Silva, J. M. , White, L. A. (1993). Relation to cognitive aptitudes to success in foreign language training. Military Psychology, 5, 79–93. doi: 10.1207/s15327876mp0502_1 First citation in articleCrossref, Google Scholar

  • Spearman, C. (1904). “General Intelligence”, objectively determined and measured. American Journal of Psychology, 15, 201–293. First citation in articleCrossref, Google Scholar

  • Stanhope, D. S. , Pond, S. , Surface, E. A. (2013). Core self-evaluations and training effectiveness: Prediction through motivational intervening mechanisms. Journal of Applied Psychology, 98, 820–831. doi: 10.1037/a0032599 First citation in articleCrossref, Google Scholar

  • Surface, E. A. (2012). Training needs assessment: Aligning learning and capability with performance requirements and organizational objectives. In M. A. Wilson, W. Bennett, Jr. , S. G. Gibson, G. M. Alliger, (Eds.), The handbook of work analysis: Methods, systems, applications and science of work measurement in organizations (pp. 437–462). New York, NY: Routledge. First citation in articleGoogle Scholar

  • Swender, E. (2003). Oral proficiency testing in the real world: Answers to frequently asked questions. Foreign Language Annals, 36, 520–526. First citation in articleCrossref, Google Scholar

  • Tonidandel, S. , LeBreton, J. M. (2011). Relative importance analysis: A useful supplement to regression analysis. Journal of Business and Psychology, 26, 1–9. doi: 10.1007/s10869-010-9204-3 First citation in articleCrossref, Google Scholar

  • US Department of Defense . (2006). Quadrennial defense review report. Washington, DC: Government Printing Office. First citation in articleGoogle Scholar

  • Van Iddekinge, C. H. , Ployhart, R. E. (2008). Developments in the criterion-related validation of selection procedures: A critical review and recommendations for practice. Personnel Psychology, 61, 871–925. doi: 10.1111/j.1744-6570.2008.00133.x First citation in articleCrossref, Google Scholar

  • Viswesvaran, C. , Ones, D. S. (2002). Agreements and disagreements on the role of general mental ability (GMA) in industrial, work, and organizational psychology. Human Performance, 15, 212–231. doi: 10.1207/S15327043HUP1501&02_13 First citation in articleCrossref, Google Scholar

  • Yung, Y. , Thissen, D. , McLeod, L. D. (1999). On the relationship between the higher-order factor model and the hierarchical factor model. Psychometrika, 64, 113–128. doi: 10.1007/BF02294531 First citation in articleCrossref, Google Scholar

  • Ziegler, M. , Dietl, E. , Danay, E. , Vogel, M. , Bühner, M. (2011). Predicting training success with general mental ability, specific ability tests, and (un)structured interviews: A meta‐analysis with unique samples. International Journal of Selection and Assessment, 19, 170–182. doi: 10.1111/j.1468-2389.2011.00544.x First citation in articleCrossref, Google Scholar