Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Multiple Speed Assessments

Theory, Practice, and Research Evidence

Published Online:December 19, 2018https://doi.org/10.1027/1015-5759/a000512

Abstract

Abstract. This paper presents Multiple Speed Assessments as an umbrella term to encompass a variety of approaches that include multiple (e.g., 20), short (e.g., 3 min), and often integrated interpersonal simulations to elicit overt behavior in a standardized way across participants. Multiple Speed Assessments can be used to get insight into the behavioral repertoire of a target person in situations sampled from a predefined target domain and their intraindividual variability across these situations. This paper outlines the characteristics and theoretical basis of Multiple Speed Assessments. We also discuss various already existing examples of Multiple Speed Assessments (Objective Structured Clinical Examinations, Multiple Mini-Interviews, and constructed response multimedia tests) and provide an overview of design variations. Finally, we present current research evidence and future research directions related to Multiple Speed Assessments. Although we present Multiple Speed Assessments in the context of personnel selection, it can also be used for assessment in the educational, personality, or clinical psychology field.

References

  • Ambady, N., Bernieri, F. J., & Richeson, J. A. (2000). Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream. Advances in Experimental Social Psychology, 32, 201–271. https://doi.org/10.1016/S0065-2601(00)80006-4 First citation in articleGoogle Scholar

  • Ambady, N., & Rosenthal, R. (1992). Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. Psychological Bulletin, 111, 256–274. https://doi.org/10.1037/0033-2909.111.2.256 First citation in articleCrossref, Google Scholar

  • Baard, S. K., Rench, T. A., & Kozlowski, S. W. J. (2014). Performance adaptation: A theoretical integration and review. Journal of Management, 40, 48–99. https://doi.org/10.1177/0149206313488210 First citation in articleCrossref, Google Scholar

  • Back, M. D., & Nestler, S. (2016). Accuracy of judging personality. In J. A. HallM. Schmid MastT. V. WestEds., The social psychology of perceiving others accurately (pp. 98–124). Cambridge, UK: Cambridge University Press. First citation in articleGoogle Scholar

  • Back, M. D., Schmukle, S. C., & Egloff, B. (2009). Predicting actual behavior from the explicit and implicit self-concept of personality. Journal of Personality and Social Psychology, 97, 533–548. https://doi.org/10.1037/a0016229 First citation in articleCrossref, Google Scholar

  • Barrick, M. R., Dustin, S. L., Giluk, T. L., Stewart, G. L., Shaffer, J. A., & Swider, B. W. (2012). Candidate characteristics driving initial impressions during rapport building: Implications for employment interview validity. Journal of Occupational and Organizational Psychology, 85, 330–352. https://doi.org/10.1111/j.2044-8325.2011.02036.x First citation in articleCrossref, Google Scholar

  • Barrick, M. R., Swider, B. W., & Stewart, G. L. (2010). Initial evaluations in the interview: Relationships with subsequent interviewer evaluations and employment offers. Journal of Applied Psychology, 95, 1163–1172. https://doi.org/10.1037/a0019918 First citation in articleCrossref, Google Scholar

  • Borkenau, P., Mauer, N., Riemann, R., Spinath, F. M., & Angleitner, A. (2004). Thin slices of behavior as cues of personality and Intelligence. Journal of Personality and Social Psychology, 86, 599–614. https://doi.org/10.1037/0022-3514.86.4.599 First citation in articleCrossref, Google Scholar

  • Brannick, M. T. (2008). Back to basics of test construction and scoring. Industrial and Organizational Psychology: Perspectives on Science and Practice, 1, 131–133. https://doi.org/10.1111/j.1754-9434.2007.00025.x First citation in articleCrossref, Google Scholar

  • Brannick, M. T., Erol-Korkmaz, H. T., & Prewett, M. (2011). A systematic review of the reliability of objective structured clinical examination scores. Medical Education, 45, 1181–1189. https://doi.org/10.1111/j.1365-2923.2011.04075.x First citation in articleCrossref, Google Scholar

  • Breil, S. M., Geukes, K., & Back, M. D. (2017). Using situational judgment tests and assessment centres in personality psychology: Three suggestions. European Journal of Personality, 31, 442–443. First citation in articleGoogle Scholar

  • Byham, W. (2016, October). Assessment centers for large populations. Presented at the International Congress on Assessment Center Methods, Bali, Indonesia. First citation in articleGoogle Scholar

  • Cameron, A. J., & MacKeigan, L. D. (2012). Development and pilot testing of a multiple mini-interview for admission to a pharmacy degree program. American Journal of Pharmaceutical Education, 76, 10. https://doi.org/10.5688/ajpe76110 First citation in articleCrossref, Google Scholar

  • Carney, D. R., Colvin, C. R., & Hall, J. A. (2007). A thin slice perspective on the accuracy of first impressions. Journal of Research in Personality, 41, 1054–1072. https://doi.org/10.1016/j.jrp.2007.01.004 First citation in articleCrossref, Google Scholar

  • Cascio, W. F., & Aguinis, H. (2008). Staffing twenty‐first‐century organizations. The Academy of Management Annals, 2, 133–165. https://doi.org/10.1080/19416520802211461 First citation in articleCrossref, Google Scholar

  • Casey, P. M., Goepfert, A. R., Espey, E. L., Hammoud, M. M., Kaczmarczyk, J. M., Katz, N. T., … Peskin, E. (2009). To the point: Reviews in medical education – the Objective Structured Clinical Examination. American Journal of Obstetrics and Gynecology, 200, 25–34. https://doi.org/10.1016/j.ajog.2008.09.878 First citation in articleCrossref, Google Scholar

  • Costa, P. Jr., Terracciano, A., & McCrae, R. R. (2001). Gender differences in personality traits across cultures: Robust and surprising findings. Journal of Personality and Social Psychology, 81, 322–331. https://doi.org/10.1037/0022-3514.81.2.322 First citation in articleCrossref, Google Scholar

  • Cucina, J. M., Su, C., Busciglio, H. H., Harris Thomas, P., & Thompson Peyton, S. (2015). Video-based testing: A high-fidelity job simulation that demonstrates reliability, validity, and utility. International Journal of Selection and Assessment, 23, 197–209. https://doi.org/10.1111/ijsa.12108 First citation in articleCrossref, Google Scholar

  • De Soete, B., Lievens, F., Oostrom, J., & Westerveld, L. (2013). Alternative predictors for dealing with the diversity–validity dilemma in personnel selection: The constructed response multimedia test. International Journal of Selection and Assessment, 21, 239–250. https://doi.org/10.1111/ijsa.12034 First citation in articleCrossref, Google Scholar

  • DeRue, D. S., Ashford, S. J., & Myers, C. G. (2012). Learning agility: In search of conceptual clarity and theoretical grounding. Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 258–279. https://doi.org/10.1111/j.1754-9434.2012.01444.x First citation in articleCrossref, Google Scholar

  • Eisenkraft, N. (2013). Accurate by way of aggregation. Journal of Experimental Social Psychology, 49, 277–279. https://doi.org/10.1016/j.jesp.2012.11.005 First citation in articleCrossref, Google Scholar

  • Epstein, S. (1979). The stability of behavior: I. On predicting most of the people much of the time. Journal of Personality and Social Psychology, 37, 1097–1126. https://doi.org/10.1037/0022-3514.37.7.1097 First citation in articleCrossref, Google Scholar

  • Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004). An admissions OSCE: The multiple mini-interview. Medical Education, 38, 314–326. https://doi.org/10.1046/j.1365-2923.2004.01776.x First citation in articleCrossref, Google Scholar

  • Feingold, A. (1994). Gender differences in personality: A meta-analysis. Psychological Bulletin, 116, 429–456. https://doi.org/10.1037/0033-2909.116.3.429 First citation in articleCrossref, Google Scholar

  • Fetzer, M. (2015). Serious games for talent selection and development. The Industrial-Organizational Psychologist, 52, 117–125. First citation in articleGoogle Scholar

  • Fleeson, W., & Gallagher, P. (2009). The implications of Big Five standing for the distribution of trait manifestation in behavior: Fifteen experience-sampling studies and a meta-analysis. Journal of Personality and Social Psychology, 97, 1097–1114. https://doi.org/10.1037/a0016786 First citation in articleCrossref, Google Scholar

  • Fleeson, W., & Jayawickreme, E. (2015). Whole trait theory. Journal of Research in Personality, 56, 82–92. https://doi.org/10.1016/j.jrp.2014.10.009 First citation in articleCrossref, Google Scholar

  • Funder, D. C. (2012). Accurate personality judgment. Current Directions in Psychological Science, 21, 177–182. https://doi.org/10.1177/0963721412445309 First citation in articleCrossref, Google Scholar

  • Funder, D. C., & Colvin, C. R. (1991). Explorations in behavioral consistency: Properties of persons, situations, and behaviors. Journal of Personality and Social Psychology, 60, 773–794. First citation in articleCrossref, Google Scholar

  • Harden, R. M., Stevenson, M., Downie, W. W., & Wilson, G. M. (1975). Assessment of clinical competence using objective structured examination. British Medical Journal, 1, 447–451. https://doi.org/10.1136/bmj.1.5955.447 First citation in articleCrossref, Google Scholar

  • Hirschmüller, S., Egloff, B., Schmukle, S. C., Nestler, S., & Back, M. D. (2015). Accurate judgments of neuroticism at zero acquaintance: A question of relevance. Journal of Personality, 83, 221–228. https://doi.org/10.1111/jopy.12097 First citation in articleCrossref, Google Scholar

  • Husbands, A., Rodgerson, M. J., Dowell, J., & Patterson, F. (2015). Evaluating the validity of an integrity-based situational judgement test for medical school admissions. BMC Medical Education, 15, 144. https://doi.org/10.1186/s12909-015-0424-0 First citation in articleCrossref, Google Scholar

  • Ingold, P. V., Dönni, M., & Lievens, F. (2018). A dual-process theory perspective to better understand judgments in assessment centers: The role of initial impressions for dimension ratings and validity. Journal of Applied Psychology. Advance online publication. https://doi.org/10.1037/apl0000333 First citation in articleCrossref, Google Scholar

  • Jackson, D. J. R., Michaelides, G., Dewberry, C., & Kim, Y.-J. (2016). Everything that you have ever been told about assessment center ratings is confounded. Journal of Applied Psychology, 101, 976–994. https://doi.org/10.1037/apl0000102 First citation in articleCrossref, Google Scholar

  • Johnston, A. N. B., Weeks, B., Shuker, M.-A., Coyne, E., Niall, H., Mitchell, M., & Massey, D. (2017). Nursing students’ perceptions of the Objective Structured Clinical Examination: An integrative review. Clinical Simulation in Nursing, 13, 127–142. https://doi.org/10.1016/j.ecns.2016.11.002 First citation in articleCrossref, Google Scholar

  • Jundt, D. K., Shoss, M. K., & Huang, J. L. (2015). Individual adaptive performance in organizations: A review. Journal of Organizational Behavior, 36, S53–S71. https://doi.org/10.1002/job.1955 First citation in articleCrossref, Google Scholar

  • Kiesler, D. J. (1983). The 1982 interpersonal circle: A taxonomy for complementarity in human transactions. Psychological Review, 90, 185–214. https://doi.org/10.1037/0033-295X.90.3.185 First citation in articleCrossref, Google Scholar

  • Knorr, M., & Hissbach, J. (2014). Multiple mini-interviews: same concept, different approaches. Medical Education, 48, 1157–1175. https://doi.org/10.1111/medu.12535 First citation in articleCrossref, Google Scholar

  • Kuncel, N. R., & Sackett, P. R. (2014). Resolving the assessment center construct validity problem (as we know it). Journal of Applied Psychology, 99, 38–47. https://doi.org/10.1037/a0034147 First citation in articleCrossref, Google Scholar

  • Leikas, S., Lönnqvist, J.-E., & Verkasalo, M. (2014). Persons, situations, and behaviors: Consistency and variability of different behaviors in four interpersonal situations. Journal of Personality and Social Psychology, 103, 1007–1022. https://doi.org/10.1037/a0030385 First citation in articleCrossref, Google Scholar

  • Leising, D., & Bleidorn, W. (2011). Which are the basic meaning dimensions of observable interpersonal behavior? Personality and Individual Differences, 51, 986–990. https://doi.org/10.1016/j.paid.2011.08.003 First citation in articleCrossref, Google Scholar

  • Levashina, J., Hartwell, C. J., Morgeson, F. P., & Campion, M. A. (2014). The structured employment interview: Narrative and quantitative review of the research literature. Personnel Psychology, 67, 241–293. https://doi.org/10.1111/peps.12052 First citation in articleCrossref, Google Scholar

  • Lievens, F. (1998). Factors which improve the construct validity of assessment centers: A review. International Journal of Selection and Assessment, 6, 141–152. https://doi.org/10.1111/1468-2389.00085 First citation in articleCrossref, Google Scholar

  • Lievens, F. (2017). Assessing personality–situation interplay in personnel selection: Toward more integration into personality research. European Journal of Personality, 31, 424–440. https://doi.org/10.1002/per.2111 First citation in articleCrossref, Google Scholar

  • Lievens, F., De Corte, W., & Westerveld, L. (2015). Understanding the building blocks of selection procedures: Effects of response fidelity on performance and validity. Journal of Management, 41, 1604–1627. https://doi.org/10.1177/0149206312463941 First citation in articleCrossref, Google Scholar

  • Lievens, F., Lang, J. W. B., De Fruyt, F., Corstjens, J., Van de Vijver, M., & Bledow, R. (2018). The predictive power of people’s intraindividual variability across situations: Implementing whole trait theory in assessment. Journal of Applied Psychology, 103, 753–771. https://doi.org/10.1037/apl0000280 First citation in articleCrossref, Google Scholar

  • Lievens, F., & Sackett, P. R. (2017). The effects of predictor method factors on selection outcomes: A modular approach to personnel selection procedures. Journal of Applied Psychology, 102, 43–66. https://doi.org/10.1037/apl0000160 First citation in articleCrossref, Google Scholar

  • Lievens, F., Schollaert, E., & Keen, G. (2015). The interplay of elicitation and evaluation of trait-expressive behavior: Evidence in assessment center exercises. Journal of Applied Psychology, 100, 1169–1188. https://doi.org/10.1037/apl0000004 First citation in articleCrossref, Google Scholar

  • Lievens, F., Tett, R. P., & Schleicher, D. J. (2009). Assessment centers at the crossroads: Toward a reconceptualization of assessment center exercises. In J. J. MartocchioH. LiaoEds., Research in personnel and human resources management (Vol. 28, pp. 99–152). Bingley, UK: JAI Press. First citation in articleGoogle Scholar

  • Meyer, R. D., Dalal, R. S., & Hermida, R. (2010). A review and synthesis of situational strength in the organizational sciences. Journal of Management, 36, 121–140. https://doi.org/10.1177/0149206309349309 First citation in articleCrossref, Google Scholar

  • Mischel, W., & Shoda, Y. (1995). A cognitive-affective system theory of personality: Reconceptualizing situations, dispositions, dynamics, and invariance in personality structure. Psychological Review, 102, 246–268. https://doi.org/10.1037/0033-295X.102.2.246 First citation in articleCrossref, Google Scholar

  • Murphy, N. A., Hall, J. A., Schmid Mast, M., Ruben, M. A., Frauendorfer, D., Blanch-Hartigan, D., … Nguyen, L. (2015). Reliability and validity of nonverbal thin slices in social interactions. Personality and Social Psychology Bulletin, 41, 199–213. https://doi.org/10.1177/0146167214559902 First citation in articleCrossref, Google Scholar

  • Oliver, T., Hausdorf, P., Lievens, F., & Conlon, P. (2016). Interpersonal dynamics in assessment center exercises: Effects of role player portrayed disposition. Journal of Management, 42, 1992–2017. https://doi.org/10.1177/0149206314525207 First citation in articleCrossref, Google Scholar

  • Oliver, T., & Lievens, F. (2014). Conceptualizing and assessing interpersonal adaptability. In D. ChanEd., Individual adaptability to changes at work: New directions in research (pp. 52–72). New York, NY: Taylor & Francis. First citation in articleGoogle Scholar

  • Oostrom, J. K., Born, M. P., Serlie, A. W., & van der Molen, H. T. (2010). Webcam testing: Validation of an innovative open-ended multimedia test. European Journal of Work and Organizational Psychology, 19, 532–550. https://doi.org/10.1080/13594320903000005 First citation in articleCrossref, Google Scholar

  • Oostrom, J. K., Born, M. P., Serlie, A. W., & van der Molen, H. T. (2011). A multimedia situational test with a constructed-response format: Its relationship with personality, cognitive ability, job experience, and academic performance. Journal of Personnel Psychology, 10, 78–88. https://doi.org/10.1027/1866-5888/a000035 First citation in articleLink, Google Scholar

  • Parrigon, S., Woo, S. E., Tay, L., & Wang, T. (2017). CAPTION-ing the situation: A lexically-derived taxonomy of psychological situation characteristics. Journal of Personality and Social Psychology, 112, 642–681. https://doi.org/10.1037/pspp0000111 First citation in articleCrossref, Google Scholar

  • Patrício, M. F., Julião, M., Fareleira, F., & Carneiro, A. V. (2013). Is the OSCE a feasible tool to assess competencies in undergraduate medical education? Medical Teacher, 35, 503–514. https://doi.org/10.3109/0142159X.2013.774330 First citation in articleCrossref, Google Scholar

  • Patterson, F., Tiffin, P. A., Lopes, S., & Zibarras, L. (2018). Unpacking the dark variance of differential attainment on examinations in overseas graduates. Medical Education, 52, 736–746. https://doi.org/10.1111/medu.13605 First citation in articleCrossref, Google Scholar

  • Patterson, F., Rowett, E., Hale, R., Grant, M., Roberts, C., Cousans, F., & Martin, S. (2016). The predictive validity of a situational judgement test and multiple-mini interview for entry into postgraduate training in Australia. BMC Medical Education, 16, 87. https://doi.org/10.1186/s12909-016-0606-4 First citation in articleCrossref, Google Scholar

  • Pau, A., Jeevaratnam, K., Chen, Y. S., Fall, A. A., Khoo, C., & Nadarajah, V. D. (2013). The Multiple Mini-Interview (MMI) for student selection in health professions training – a systematic review. Medical Teacher, 35, 1027–1041. https://doi.org/10.3109/0142159X.2013.829912 First citation in articleCrossref, Google Scholar

  • Putka, D. J., & Hoffman, B. J. (2013). Clarifying the contribution of assessee-, dimension-, exercise-, and assessor-related effects to reliable and unreliable variance in assessment center ratings. Journal of Applied Psychology, 98, 114–133. https://doi.org/10.1037/a0030887 First citation in articleCrossref, Google Scholar

  • Rauthmann, J., Gallardo-Pujol, D., Guillaume, E., Todd, E., Nave, C., Sherman, R. A., … Funder, D. C. (2014). The situational eight DIAMONDS: A taxonomy of major dimensions of situation characteristics. Personality Processes and Individual Differences, 107, 677–718. https://doi.org/10.1037/a0037250 First citation in articleGoogle Scholar

  • Rees, E. L., Hawarden, A. W., Dent, G., Hays, R., Bates, J., & Hassell, A. B. (2016). Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37. Medical Teacher, 38, 443–455. https://doi.org/10.3109/0142159X.2016.1158799 First citation in articleCrossref, Google Scholar

  • Roberts, B. W., Luo, J., Briley, D. A., Chow, P. I., Su, R., & Hill, P. L. (2017). A systematic review of personality trait change through intervention. Psychological Bulletin, 143, 117–141. https://doi.org/10.1037/bul0000088 First citation in articleCrossref, Google Scholar

  • Roberts, C., Clark, T., Burgess, A., Frommer, M., Grant, M., & Mossman, K. (2014). The validity of a behavioural multiple-mini-interview within an assessment centre for selection into specialty training. BMC Medical Education, 14, 169. https://doi.org/10.1186/1472-6920-14-169 First citation in articleCrossref, Google Scholar

  • Roch, S. G., Woehr, D. J., Mishra, V., & Kieszczynska, U. (2012). Rater training revisited: An updated meta-analytic review of frame-of-reference training. Journal of Occupational and Organizational Psychology, 85, 370–395. https://doi.org/10.1111/j.2044-8325.2011.02045.x First citation in articleCrossref, Google Scholar

  • Roth, P. L., Bobko, P., Van Iddekinge, C. H., & Thatcher, J. B. (2016). Social media in employee-selection-related decisions: A research agenda for uncharted territory. Journal of Management, 42, 269–298. https://doi.org/10.1177/0149206313503018 First citation in articleCrossref, Google Scholar

  • Rushforth, H. E. (2007). Objective structured clinical examination (OSCE): Review of literature and implications for nursing education. Nurse Education Today, 27, 481–490. https://doi.org/10.1016/j.nedt.2006.08.009 First citation in articleCrossref, Google Scholar

  • Sackett, P. R., Zedeck, S., & Fogli, L. (1988). Relations between measures of typical and maximum job performance. Journal of Applied Psychology, 73, 482–486. First citation in articleCrossref, Google Scholar

  • Schmitt, N., & Ostroff, C. (1986). Operationalizing the “behavioral consistency” approach: Selection test development based on a content-oriented strategy. Personnel Psychology, 39, 91–108. https://doi.org/10.1111/j.1744-6570.1986.tb00576.x First citation in articleCrossref, Google Scholar

  • Schollaert, E., & Lievens, F. (2011). The use of role-player prompts in assessment center exercises. International Journal of Selection and Assessment, 19, 190–197. https://doi.org/10.1111/j.1468-2389.2011.00546.x First citation in articleCrossref, Google Scholar

  • Schollaert, E., & Lievens, F. (2012). Building situational stimuli in assessment center exercises: Do specific exercise instructions and role-player prompts increase the observability of behavior? Human Performance, 25, 255–271. https://doi.org/10.1080/08959285.2012.683907 First citation in articleCrossref, Google Scholar

  • Tett, R. P., & Burnett, D. D. (2003). A personality trait-based interactionist model of job performance. Journal of Applied Psychology, 88, 500–517. https://doi.org/10.1037/0021-9010.88.3.500 First citation in articleCrossref, Google Scholar

  • Tiller, D., O’Mara, D., Rothnie, I., Dunn, S., Lee, L., & Roberts, C. (2013). Internet-based multiple mini-interviews for candidate selection for graduate entry programmes. Medical Education, 47, 801–810. https://doi.org/10.1111/medu.12224 First citation in articleCrossref, Google Scholar

  • Van der Vleuten, C. P. M., & Swanson, D. B. (1990). Assessment of clinical skills with standardized patients: State of the art. Teaching and Learning in Medicine, 2, 58–76. https://doi.org/10.1080/10401339009539432 First citation in articleCrossref, Google Scholar

  • Wang, L., & Grimm, K. J. (2012). Investigating reliabilities of intraindividual variability indicators. Multivariate Behavioral Research, 47, 771–802. https://doi.org/10.1080/00273171.2012.715842 First citation in articleCrossref, Google Scholar

  • Wernimont, P. F., & Campbell, J. P. (1968). Signs, samples, and criteria. Journal of Applied Psychology, 52, 372. https://doi.org/10.1037/h0026244 First citation in articleCrossref, Google Scholar

  • Wood, T. J., Chan, J., Humphrey-Murto, S., Pugh, D., & Touchie, C. (2017). The influence of first impressions on subsequent ratings within an OSCE station. Advances in Health Sciences Education, 22, 969–983. https://doi.org/10.1007/s10459-016-9736-z First citation in articleCrossref, Google Scholar

  • Woolf, K., Haq, I., McManus, I. C., Higham, J., & Dacre, J. (2008). Exploring the underperformance of male and minority ethnic medical students in first year clinical examinations. Advances in Health Sciences Education, 13, 607–616. https://doi.org/10.1007/s10459-007-9067-1 First citation in articleCrossref, Google Scholar

  • Yukl, G. (1989). Leadership in organizations (2nd ed.). Englewood Cliffs, NJ: Prentice Hall. First citation in articleGoogle Scholar

  • Ziegler, M. (2014). B5PS. Big Five Inventory of personality in occupational situations. Mödling, Austria: Schuhfried GmbH. First citation in articleGoogle Scholar