Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Toward Stable Predictions of Apprentices’ Training Success

Can Artificial Neural Networks Outperform Linear Predictions?

Published Online:March 23, 2011https://doi.org/10.1027/1866-5888/a000027

Abstract

Mechanical (statistical) predictions have proven to be useful in personnel selection. However, such predictions require the use of an algorithm to aggregate different predictor scores. The identification of such an algorithm requires analyzing predictor and criterion data obtained from previous applicants. The present manuscript compared predictions made by two different statistical methods: artificial neural networks (ANNs) and multiple regression analysis. Therefore, three consecutive cohorts of apprentices (n = 322, 217, and 118) were examined. Algorithms were derived from one cohort and applied to more recent cohorts. It was shown that ANNs outperformed linear predictions in a cross-validation within the cohorts. However, applying trained ANNs to other samples resulted in a predictive power which was worse than most of the linear predictions. Thus, we conclude that ANNs should only be used as selection algorithm if their validity in different cohorts has been confirmed.

References

  • Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B. N. Petrov, F. Csaki (Eds.), Proceedings of the 2nd International Symposium on Information Theory (pp. 267–281). Budapest: Akademiai Kiado. First citation in articleGoogle Scholar

  • Arendasy, M. , Sommer, M. , Hergovich, A. (2007). Statistical judgment formation in personnel selection: A study in military aviation psychology. Military Psychology, 19, 119–136. First citation in articleCrossref, Google Scholar

  • Bennett, G. K. (1969). Mechanical Comprehension Test (MCT). New York, NY: The Psychological Corporation. First citation in articleGoogle Scholar

  • Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford, UK: Oxford University Press. First citation in articleGoogle Scholar

  • Borack, J. I. (1995). Alternative techniques for predicting success in air controller school. Military Psychology, 7, 207–219. First citation in articleCrossref, Google Scholar

  • Carson, A. , Bizot, E. , Kraemer, B. (1999). Modeling career counselor decisions with artificial neural networks: Predictions of fit across a comprehensive occupational map. Journal of Vocational Behavior, 54, 196–207. First citation in articleCrossref, Google Scholar

  • Cohen, J. , Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Collins, J. , Clark, M. R. (1993). An application of the theory of neural computation to the prediction of workplace behavior: An illustration and assessment of network analysis. Personnel Psychology, 46, 503–524. First citation in articleCrossref, Google Scholar

  • Funahashi, K. (1989). On the approximate realization of continuous mappings by neural networks. Neural Networks, 2, 183–192. First citation in articleCrossref, Google Scholar

  • Griffin, R. B. (1998). Predicting naval aviator flight training performance using multiple regression and artificial neural networks. International Journal of Aviation Psychology, 8, 121–135. First citation in articleCrossref, Google Scholar

  • Grove, W. M. , Zald, D. H. , Lebow, B. S. , Snitz, B. E. , Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12, 19–30. First citation in articleCrossref, Google Scholar

  • Haeusler, J. , Sommer, M. (2006). Neuronale Netze: Nichtlineare Methoden der statistischen Urteilsbildung in der psychologischen Eignungsdiagnostik [Neural networks: Nonlinear methods of judgment formation in personnel selection]. Zeitschrift fuer Personalpsychologie, 5, 4–15. First citation in articleLink, Google Scholar

  • Hülsheger, U. R. , Maier, G. W. , Stumpp, T. , & Muck, P. M (2006). Vergleich kriteriumsbezogener Validitäten verschiedener Intelligenztests zur Vorhersage von Ausbildungserfolg in Deutschland. Ergebnisse einer Metaanalyse [Comparing criterion-related validities of different intelligence tests for the prediction of training success in Germany: A meta-analysis]. Zeitschrift für Personalpsychologie, 5, 145–162. First citation in articleLink, Google Scholar

  • Jaeger, A. O. , Althoff, K. (1983). Wilde Intelligenz Test: Ein Strukturdiagnostikum [Wilde Intelligence Test: A diagnostic tool for intelligence structure]. Göttingen, Germany: Hogrefe. First citation in articleGoogle Scholar

  • Kahneman, D. , Slovic, P. , Tversky, A. (1982). Judgement under uncertainty: Heuristics and biases. Cambridge, UK: Cambridge University Press. First citation in articleCrossref, Google Scholar

  • Kersting, M. , Hornke, L. F. (2006). Improving the quality of proficiency assessment: The German standardization approach. Psychology Science, 48, 85–98. First citation in articleGoogle Scholar

  • Kinnebrock, W. (1992). Neuronale Netze. Grundlagen, Anwendungen, Beispiele [Neural networks. Basics, applications, examples]. München, Germany: Oldenbourg. First citation in articleGoogle Scholar

  • LaHuis, D. M. , Martin, N. R. , Avis, J. M. (2005). Investigating nonlinear conscientiousness-job performance relations for clerical employees. Human Performance, 18, 199–212. First citation in articleCrossref, Google Scholar

  • Maukisch, H. (1990). Eignungsdiagnostik als prognostische Hilfe bei der Auswahl, Platzierung und Entwicklung von Führungskräften [Proficiency assessment as a prediction tool in selection, placement, and development of managerial staff]. In W. Sarges (Ed.) Management Diagnostik (pp. 32–46). Göttingen, Germany: Hogrefe. First citation in articleGoogle Scholar

  • Motowidlo, S. J. , Brownlee, A. L. , Schmit, M. J. (2008). Effects of personality characteristics on knowledge, skill, and performance in servicing retail customers. International Journal of Selection and Assessment, 16, 272–280. First citation in articleCrossref, Google Scholar

  • Ng, T. W. H. , Feldman, D. C. (2008). The relationship of age to ten dimensions of job performance. Journal of Applied Psychology, 93, 392–423. First citation in articleCrossref, Google Scholar

  • Poddig, T. , Sidorovitch, I. (2001). Künstliche Neuronale Netze: Überblick, Einsatzmöglichkeiten und Anwendungsprobleme. In H. Hippner, U. Küsters, M. Meyer, K. Wilde (Eds.), Handbuch Data Mining im Marketing: Knowledge discovery in marketing databases. Braunschweig, Germany: Vieweg. First citation in articleGoogle Scholar

  • Price, R. K. , Spitznagel, E. L. , Downey, T. J. , Meyer, D. J. , Risk, N. K. , & El-Ghazzawy, O. G. (2000). Applying artificial neural network models to clinical decision making. Psychological Assessment, 12, 40–51. First citation in articleCrossref, Google Scholar

  • R-Development-Core-Team (2006). R: A language and environment for statistical computing (version 2.3.1). Vienna, Austria: R Foundation for Statistical Computing. First citation in articleGoogle Scholar

  • Rakes, T. (1991). Research in the neural networks/expert system area. Decision Line, 22, 11–14. First citation in articleGoogle Scholar

  • Ree, M. J. , Earles, J. A. (1991). Predicting training success: Not much more than g. Personnel Psychology, 44, 321–332. First citation in articleCrossref, Google Scholar

  • Salgado, J. F. , Anderson, N. , Moscoso, S. , Bertua, C. , de Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European community meta-analysis. Personnel Psychology, 56, 573–605. First citation in articleCrossref, Google Scholar

  • Scarborough, D. , Somers, M. J. (2006). Neural networks in organizational research: Applying pattern recognition to the analysis of organizational behavior. Washington, DC: American Psychological Association. First citation in articleCrossref, Google Scholar

  • Schmidt, F. L. , Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274. First citation in articleCrossref, Google Scholar

  • Schmidt-Atzert, L. , Deter, B. , Jaeckel, S. (2004). Prädiktion von Ausbildungserfolg: Allgemeine Intelligenz (g) oder spezifische kognitive Fähigkeiten? [The prediction of training success: General intelligence (g) or specific cognitive abilities?]. Zeitschrift für Personalpsychologie, 3, 147–158. First citation in articleLink, Google Scholar

  • Schuhfried, G. (2005). Manual Expertensystem Verkehr [Manual for an expert system for traffic]. Mödling: Schuhfried GmbH. First citation in articleGoogle Scholar

  • Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464. First citation in articleCrossref, Google Scholar

  • Somers, M. J. (1999). Application of two neural network paradigms to the study of voluntary employee turnover. Journal of Applied Psychology, 84, 177–185. First citation in articleCrossref, Google Scholar

  • Somers, M. J. (2001). Thinking differently: Assessing nonlinearities in the relationship between work attitudes and job performance using a Bayesian neural network. Journal of Occupational and Organizational Psychology, 74, 47–61. First citation in articleCrossref, Google Scholar

  • Sommer, M. , Arendasy, M. , Schuhfried, G. , Litzenberger, M. (2005). Diagnostische Unterscheidbarkeit unfallfreier und mehrfach unfallbelasteter Kraftfahrer mit Hilfe nicht-linearer Auswertemethoden [Differentiation between accident-free drivers and drivers with multiple accidents using nonlinear statistical methods]. Zeitschrift für Verkehrssicherheit, 51, 82–86. First citation in articleGoogle Scholar

  • Sommer, M. , Olbrich, A. , & Arendasy, M. (2004). Improvements in personnel selection with neural networks: A pilot study in the field of aviation psychology. International Journal of Aviation Psychology, 14, 103–115. First citation in articleCrossref, Google Scholar

  • Venables, W. N. , Ripley, B. D. (2002). Modern applied statistics with S. New York, NY: Springer. First citation in articleCrossref, Google Scholar

  • Warner, B. , Misra, M. (1996). Understanding neural networks as statistical tools. The American Statistician, 50, 284–293. First citation in articleGoogle Scholar

  • Wright, P. M. , Kacmar, K. M. , McMahan, G. C. , Deleeuw, K. (1995). Cognitive ability as a moderator of the relationship between personality and job performance. Journal of Management, 21, 1129–1139. First citation in articleCrossref, Google Scholar