Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

A Multimedia Situational Test With a Constructed-Response Format

Its Relationship With Personality, Cognitive Ability, Job Experience, and Academic Performance

Published Online:June 02, 2011https://doi.org/10.1027/1866-5888/a000035

Abstract

Advances in computer technology have created opportunities for the development of a multimedia situational test in which responses are filmed with a webcam. This paper examined the relationship of a so-called webcam test with personality, cognitive ability, job experience, and academic performance. Data were collected among 153 psychology students. In line with our expectations, scores on the webcam test, intended to measure interpersonally oriented leadership, were related to extraversion, conscientiousness, emotional stability, and job experience. Furthermore, the webcam tests significantly predicted students’ learning activities during group meetings over and above a cognitive ability test and a personality questionnaire. Overall, this study demonstrates that webcam tests can be a valid complement to traditional predictors in selection contexts.

References

  • Arthur, W. J. , Villado, A. (2008). The importance of distinguishing between constructs and methods when comparing predictors in personnel selection research and practice. Journal of Applied Psychology, 93, 435–442. First citation in articleCrossref, Google Scholar

  • Asher, J. J. , Sciarrino, J. A. (1974). Realistic work sample tests: A review. Personnel Psychology, 27, 519–533. First citation in articleCrossref, Google Scholar

  • Attali, Y. (2005). Reliability of speeded number-right multiple-choice tests. Applied Psychological Measurement, 29, 357–368. First citation in articleCrossref, Google Scholar

  • Bergman, M. E. , Drasgow, F. , Donovan, M. A. , Henning, J. B. , Juraska, S. E. (2006). Scoring situational judgment tests: Once you get the data, your troubles begin. International Journal of Selection and Assessment, 14, 223–235. First citation in articleCrossref, Google Scholar

  • Bernardin, H. J. , Buckley, M. R. (1981). Strategies in rater training. Academy of Management Review, 6, 205–212. First citation in articleGoogle Scholar

  • Campbell, D. T. , Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105. First citation in articleCrossref, Google Scholar

  • Chan, D. , Schmitt, N. (2005). Situational judgment tests. In A. Evers, O. Smit-Voskuil, N. Anderson (Eds.), Handbook of personnel selection (pp. 219–246). Oxford, UK: Blackwell. First citation in articleGoogle Scholar

  • Christian, M. S. , Edwards, J. C. , Bradley, J. C. (2010). Situational judgment tests: Constructs assessed and a meta-analysis of their criterion-related validities. Personnel Psychology, 63, 83–117. First citation in articleCrossref, Google Scholar

  • Clevenger, J. , Pereira, G. M. , Wiechmann, D. , Schmitt, N. , Harvey, V. S. (2001). Incremental validity of situational judgment tests. Journal of Applied Psychology, 86, 410–417. First citation in articleCrossref, Google Scholar

  • Costa, P. T. , McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI). Odessa, FL: Psychological Assessment Resources. First citation in articleGoogle Scholar

  • Cronbach, L. J. , Gleser, G. C. (1965). Psychological tests and personnel decisions. Urbana, IL: University of Illinois Press. First citation in articleGoogle Scholar

  • Drenth, P. J. D. (1965). Test voor niet-verbale abstractie. Amsterdam: Swets & Zeitlinger. First citation in articleGoogle Scholar

  • Funke, U. , Schuler, H. (1998). Validity of stimulus and response components in a video test of social competence. International Journal of Selection and Assessment, 6, 115–123. First citation in articleCrossref, Google Scholar

  • Gaugler, B. B. , Rosenthal, D. B. , Thornton, G. C. , Bentson, C. (1987). Meta-analysis of assessment center validity. Journal of Applied Psychology, 72, 493–511. First citation in articleCrossref, Google Scholar

  • Goldberg, L. R. (1990). An alternative description of personality: The Big-Five factor structure. Journal of Personality and Social Psychology, 59, 1216–1229. First citation in articleCrossref, Google Scholar

  • Griffin, B. , Hesketh, B. (2004). Why openness to experience is not a good predictor of job performance. International Journal of Selection and Assessment, 12, 243–251. First citation in articleCrossref, Google Scholar

  • Hu, L. , Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. First citation in articleCrossref, Google Scholar

  • Hunter, J. E. , Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72–98. First citation in articleCrossref, Google Scholar

  • Judge, T. A. , Bono, J. E. , Ilies, R. , & Gerhardt, M. W. (2002). Personality and leadership: A qualitative and quantitative review. Journal of Applied Psychology, 87, 765–778. First citation in articleCrossref, Google Scholar

  • Koch, B. P. N. (1998). Handleiding G5. Unpublished manuscript. First citation in articleGoogle Scholar

  • Latham, G. P. , Saari, L. M. , Pursell, E. D. , Campion, M. A. (1980). The situational interview. Journal of Applied Psychology, 65, 422–427. First citation in articleCrossref, Google Scholar

  • Lievens, F. (1998). Factors which improve the construct validity of assessment centers: A review. International Journal of Selection and Assessment, 6, 141–152. First citation in articleCrossref, Google Scholar

  • Lievens, F. , Buyse, T. , Sackett, P. R. (2005). The operational validity of a video-based situational judgment test for medical college admissions: Illustrating the importance of matching predictor and criterion construct domains. Journal of Applied Psychology, 90, 442–452. First citation in articleCrossref, Google Scholar

  • Lievens, F. , Peeters, H. , & Schollaert, E. (2008). Situational judgment tests: A review of recent research. Personnel Review, 37, 426–441. First citation in articleCrossref, Google Scholar

  • Lievens, F. , Sackett, P. R. (2006). Video-based versus written situational judgment tests: A comparison in terms of predictive validity. Journal of Applied Psychology, 91, 1181–1188. First citation in articleCrossref, Google Scholar

  • Lievens, F. , Thornton, G. C., III . (2005). Assessment centers: Recent developments in practice and research. In A. Evers, O. Smit-Voskuijl, N. Anderson (Eds.), Handbook of selection (pp. 243–264). London: Blackwell. First citation in articleGoogle Scholar

  • Lounsbury, J. W. , Sundstrom, E. , Loveland, J. M. , Gibson, L. W. (2003). Intelligence, “Big Five” personality traits, and work drive as predictors of course grades. Personality and Individual Differences, 35, 1231–1239. First citation in articleCrossref, Google Scholar

  • Loyens, S. M. M. , Rikers, R. M. J. P. , Schmidt, H. G. (2007). The impact of students’ conceptions of constructivist assumptions on academic achievement and drop-out. Studies in Higher Education, 32, 581–602. First citation in articleCrossref, Google Scholar

  • McCrae, R. R. (1994). Openness to experience: Expanding the boundaries of Factor V. European Journal of Personality, 8, 251–272. First citation in articleCrossref, Google Scholar

  • McDaniel, M. A. , Hartman, N. S. , Whetzel, D. L. , Grubb, W. L. III . (2007). Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology, 60, 63–91. First citation in articleCrossref, Google Scholar

  • McDaniel, M. A. , Morgeson, F. P. , Finnegan, E. B. , Campion, M. A. , Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86, 730–740. First citation in articleCrossref, Google Scholar

  • McDaniel, M. A. , Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection and Assessment, 9, 103–113. First citation in articleCrossref, Google Scholar

  • Motowidlo, S. J. , Borman, W. C. , Schmit, M. J. (1997). A theory of individual differences in task performance and contextual performance. Human Performance, 10, 71–83. First citation in articleCrossref, Google Scholar

  • Motowidlo, S. J. , Brownlee, A. L. , Schmit, M. J. (2008). Effects of personality characteristics on knowledge, skill, and performance in servicing retail customers. International Journal of Selection and Assessment, 16, 272–281. First citation in articleCrossref, Google Scholar

  • Motowidlo, S. J. , Dunnette, M. D. , Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640–647. First citation in articleCrossref, Google Scholar

  • Olson-Buchanan, J. B. , Drasgow, F. (2006). Multimedia situational judgment tests: The medium creates the message. In J. A. Weekley, R. E. Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 253–278). Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Oostrom, J. K. , Born, M. Ph. , Serlie, A. W. , Van der Molen, H. T. (2010). Webcam testing: Validation of an innovative open-ended multimedia test. European Journal of Work and Organizational Psychology, 19, 532–550. First citation in articleCrossref, Google Scholar

  • Oswald, F. L. , Schmitt, N. , Kim, B. H. , Ramsay, L. J. , Gillespie, M. A. (2004). Developing a biodata measure and situational judgment inventory as predictors of college student performance. Journal of Applied Psychology, 89, 187–207. First citation in articleCrossref, Google Scholar

  • Peterson, N. G. , Mumford, M. D. , Borman, W. C. , Jeanneret, P. R. , Fleishman, E. A. (1999). An occupational information system for the 21st century: The development of O*Net. Washington, DC: American Psychological Association. First citation in articleCrossref, Google Scholar

  • Ployhart, R. E. , Holtz, B. C. (2008). The diversity-validity dilemma: Strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology, 61, 153–172. First citation in articleCrossref, Google Scholar

  • Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135, 322–338. First citation in articleCrossref, Google Scholar

  • Schmitt, N. , Chan, D. (2006). Situational judgment tests: Method or construct?. In J. A. Weekley, R. E. Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 135–155). Erlbaum: NJ. First citation in articleGoogle Scholar

  • Schmitt, N. , Ostroff, C. (1986). Operationalizing the “behavioral consistency” approach: Selection test development based on a content-oriented strategy. Personnel Psychology, 39, 91–108. First citation in articleCrossref, Google Scholar

  • Semeijn, J. H. , Van der Velden, R. , Heijke, H. , Van der Vleuten, C. , Boshuizen, H. (2005). Competence indicators in academic education and early labour market success of graduates in health sciences, RM-2005–2E. Maastricht The Netherlands: Research Centre for Education and the Labour Market. First citation in articleGoogle Scholar

  • Sosik, J. J. , Kahai, S. S. , Avolio, B. J. (1998). Transformational leadership and dimensions of creativity: Motivating idea generation in computer-mediated groups. Creativity Research Journal, 11, 111–121. First citation in articleCrossref, Google Scholar

  • Spychalski, A. C. , Quinones, M. A. , Gaugler, B. B. , Pohley, K. (1997). A survey of assessment center practices in organizations in the United States. Personnel Psychology, 50, 71–90. First citation in articleCrossref, Google Scholar

  • Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245–251. First citation in articleCrossref, Google Scholar

  • Stemler, S. E. , Sternberg, R. J. (2006). Using situational judgment tests to measure practical intelligence. In J. A. Weekley, R. E. Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 107–131). Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Thornton, G. C. , Byham, W. C. (1982). Assessment centers and managerial performance. New York, NY: Academic Press. First citation in articleGoogle Scholar

  • Trapmann, S. , Hell, B. , Hirn, J. O. W. , Schuler, H. (2007). Meta-analysis of the relationship between the Big Five and academic success at university. Zeitschrift für Psychologie, 215, 132–151. First citation in articleLink, Google Scholar

  • Van Leeuwen, R. G. J. (2004). Handleiding Captain. Unpublished manuscript. First citation in articleGoogle Scholar

  • Weekley, J. A. , Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50, 25–49. First citation in articleCrossref, Google Scholar

  • Wernimont, P. F. , Campbell, J. P. (1968). Signs, samples, and criteria. Journal of Applied Psychology, 52, 372–376. First citation in articleCrossref, Google Scholar

  • Yukl, G. (1998). Leadership in organizations. Upper Saddle River, NJ: Prentice Hall. First citation in articleGoogle Scholar