Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Spotting the “Ideal” Personality Response

Effects of Item Matching in Forced Choice Measures for Personnel Selection

Published Online:January 27, 2021https://doi.org/10.1027/1866-5888/a000267

Abstract

Abstract. Forced choice (FC) personality questionnaires attempt to constrain job applicants from presenting idealized responses (or “faking”). FC questionnaires are designed by identifying items equally desirable in applicants, matching these into “blocks,” and instructing respondents to rank the items “most like” themselves. Nonetheless, how closely items should be matched remains unclear, and desirability seems dependent on the job. We investigated how strongly respondents (N = 436) agreed regarding the “ideal” applicant response, while varying (a) how closely items were matched into blocks and (b) the job context. While the most closely matched blocks elicited slight agreement on an ideal response, agreement increased noticeably with poorer matching. Nonetheless, differences in item desirability between different job conditions were evident even in closely matched blocks.

References

  • Allison, P. D., & Christakis, N. A. (1994). Logit models for sets of ranked items. Sociological Methodology, 24, 199–228. 10.2307/270983 First citation in articleCrossref, Google Scholar

  • Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B (Methodological), 57(1), 289–300. 10.2307/2346101 First citation in articleCrossref, Google Scholar

  • Berkshire, J. R. (1958). Comparisons of five forced-choice indices. Educational and Psychological Measurement, 18(3), 553–561. 10.1177/001316445801800309 First citation in articleCrossref, Google Scholar

  • Birkeland, S. A., Manson, T. M., Kisamore, J. L., Brannick, M. T., & Smith, M. A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14(4), 317–335. 10.1111/j.1468-2389.2006.00354.x First citation in articleCrossref, Google Scholar

  • Brown, A., & Bartram, D. (2009, April). Doing less but getting more: Improving forced-choice measures with IRT. Conference session at the 24th Annual Conference of the Society for Industrial & Organizational Psychology (SIOP), New Orleans, LA, USA. First citation in articleGoogle Scholar

  • Brown, A., & Maydeu-Olivares, A. (2014). Modeling forced-choice response formats. In P. IrwingT. BoothD. J. HughesEds., The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development (pp. 523–569). Wiley Blackwell. 10.1002/9781118489772.ch18 First citation in articleCrossref, Google Scholar

  • Burns, G., & Christiansen, N. (2011). Methods of measuring faking behavior. Human Performance, 24(4), 358–372. 10.1080/08959285.2011.597473 First citation in articleCrossref, Google Scholar

  • Cao, M., & Drasgow, F. (2019). Does forcing reduce faking? A meta-analytic review of forced-choice personality measures in high-stakes situations. Journal of Applied Psychology, 104(11), 1347–1368. 10.1037/apl0000414 First citation in articleCrossref, Google Scholar

  • Cheung, J. H., Burns, D. K., Sinclair, R. R., & Sliter, M. (2017). Amazon mechanical turk in organizational psychology: An evaluation and practical recommendations. Journal of Business and Psychology, 32(4), 347–361. 10.1007/s10869-016-9458-5 First citation in articleCrossref, Google Scholar

  • Christiansen, N. D., Burns, G. N., & Montgomery, G. E. (2005). Reconsidering forced-choice item formats for applicant personality assessment. Human Performance, 18(3), 267–307. 10.1207/s15327043hup1803_4 First citation in articleCrossref, Google Scholar

  • Converse, P. D., Pathak, J., Quist, J., Merbedone, M., Gotlib, T., & Kostic, E. (2010). Statement desirability ratings in forced-choice personality measure development: Implications for reducing score inflation and providing trait-level information. Human Performance, 23(4), 323–342. 10.1080/08959285.2010.501047 First citation in articleCrossref, Google Scholar

  • Corah, N. L., Feldman, M. J., Cohen, I. S., Gruen, W., Meadow, A., & Ringwall, E. A. (1958). Social desirability as a variable in the edwards personal preference schedule. Journal of Consulting Psychology, 22(1), 70–72. 10.1037/h0048286 First citation in articleCrossref, Google Scholar

  • DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Between facets and domains: 10 aspects of the Big Five. Journal of Personality and Social Psychology, 93(5), 880–896. 10.1037/0022-3514.93.5.880 First citation in articleCrossref, Google Scholar

  • Dueber, D. M., Love, A. M. A., Toland, M. D., & Turner, T. A. (2019). Comparison of single-response format and forced-choice format instruments using thurstonian item response theory. Educational and Psychological Measurement, 79(1), 108–128. 10.1177/0013164417752782 First citation in articleCrossref, Google Scholar

  • Dunlop, P. D., Telford, A. D., & Morrison, D. L. (2012). Not too little, but not too much: The perceived desirability of responses to personality items. Journal of Research in Personality, 46(1), 8–18. 10.1016/j.jrp.2011.10.004 First citation in articleCrossref, Google Scholar

  • Feldman, M. J., & Corah, N. L. (1960). Social desirability and the forced choice method. Journal of Consulting Psychology, 24(6), 480–482. 10.1037/h0042687 First citation in articleCrossref, Google Scholar

  • Goffin, R. D., Jang, I., & Skinner, E. (2011). Forced-choice and conventional personality assessment: Each may have unique value in pre-employment testing. Personality and Individual Differences, 51(7), 840–844. 10.1016/j.paid.2011.07.012 First citation in articleCrossref, Google Scholar

  • Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In I. MervieldeF. DearyF. De FruytF. OstendorfEds., Personality psychology in Europe (Vol. 7, pp. 7–28). Tilburg University Press. First citation in articleGoogle Scholar

  • Goldberg, L. R. (1992). The development of markers for the Big-Five factor structure. Psychological Assessment, 4(1), 26–42. 10.1037/1040-3590.4.1.26 First citation in articleCrossref, Google Scholar

  • Gordon, L. V. (1951). Validities of the forced-choice and questionnaire methods of personality measurement. Journal of Applied Psychology, 35(6), 407–412. 10.1037/h0058853 First citation in articleCrossref, Google Scholar

  • Griffith, R. L., Chmielowski, T., & Yoshita, Y. (2007). Do applicants fake? An examination of the frequency of applicant faking behavior. Personnel Review, 36(3), 341–355. 10.1108/00483480710731310 First citation in articleCrossref, Google Scholar

  • Heggestad, E. D., Morrison, M., Reeve, C. L., & McCloy, R. A. (2006). Forced-choice assessments of personality for selection: Evaluating issues of normative assessment and faking resistance. Journal of Applied Psychology, 91(1), 9–24. 10.1037/0021-9010.91.1.9 First citation in articleCrossref, Google Scholar

  • Jackson, D. N., Wroblewski, V. R., & Ashton, M. C. (2000). The impact of faking on employment tests: Does forced choice offer a solution? Human Performance, 13(4), 371–388. 10.1207/S15327043HUP1304_3 First citation in articleCrossref, Google Scholar

  • Johnson, J. A. (2005). Ascertaining the validity of individual protocols from web-based personality inventories. Journal of Research in Personality, 39(1), 103–129. 10.1016/j.jrp.2004.09.009 First citation in articleCrossref, Google Scholar

  • Kendall, M. G., & Gibbons, J. D. (1990). Rank correlation methods (5th ed.). Hodder Arnold. First citation in articleGoogle Scholar

  • Krug, R. E. (1958). A selection set preference index. Journal of Applied Psychology, 42(3), 168–170. 10.1037/h0046461 First citation in articleCrossref, Google Scholar

  • Kuncel, N. R., Goldberg, L. R., & Kiger, T. (2011). A plea for process in personality prevarication. Human Performance, 24(4), 373–378. 10.1080/08959285.2011.597476 First citation in articleCrossref, Google Scholar

  • Pavlov, G., Maydeu-Olivares, A., & Fairchild, A. J. (2018). Effects of applicant faking on forced-choice and likert scores. Organizational Research Methods, 22(3), 710–739. 10.1177/1094428117753683 First citation in articleCrossref, Google Scholar

  • R Core Team. (2019). R: A language and environment for statistical computing [Computer software]. https://www.R-project.org/ First citation in articleGoogle Scholar

  • Sackett, P. R., & Walmsley, P. T. (2014). Which personality attributes are most important in the workplace? Perspectives on Psychological Science, 9(5), 538–551. 10.1177/1745691614543972 First citation in articleCrossref, Google Scholar

  • Soto, C. J., & John, O. P. (2017). The next Big Five inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113(1), 117–143. 10.1037/pspp0000096 First citation in articleCrossref, Google Scholar

  • Stark, S., Chernyshenko, O. S., & Drasgow, F. (2011). Constructing fake-resistant personality tests using item response theory. In M. ZieglerC. MacCannR. RobertsEds., New Perspectives on Faking in Personality Assessment (pp. 214–239). Oxford University Press. 10.1093/acprof:oso/9780195387476.003.0061 First citation in articleCrossref, Google Scholar

  • Therneau, T. M. (2015). A package for survival analysis in S. https://CRAN.R-project.org/package=survival First citation in articleGoogle Scholar

  • Usami, S., Sakamoto, A., Naito, J., & Abe, Y. (2016). Developing pairwise preference-based personality test and experimental investigation of its resistance to faking effect by item response model. International Journal of Testing, 16(4), 288–309. 10.1080/15305058.2016.1145123 First citation in articleCrossref, Google Scholar

  • Vasilopoulos, N. L., Cucina, J. M., Dyomina, N. V., Morewitz, C. L., & Reilly, R. R. (2006). Forced-choice personality tests: A measure of personality and cognitive ability? Human Performance, 19(3), 175–199. 10.1207/s15327043hup1903_1 First citation in articleCrossref, Google Scholar

  • Waters, L. K. (1965). A note on the “fakability” of forced-choice scales. Personnel Psychology, 18(2), 187–191. 10.1111/j.1744-6570.1965.tb00277.x First citation in articleCrossref, Google Scholar

  • Wetzel, E., Roberts, B. W., Fraley, R. C., & Brown, A. (2016). Equivalence of narcissistic personality inventory constructs and correlates across scoring approaches and response formats. Journal of Research in Personality, 61, 87–98. 10.1016/j.jrp.2015.12.002 First citation in articleCrossref, Google Scholar

  • Zeileis, A., & Hothorn, T. (2002). Diagnostic checking in regression relationships. R News, 2(3), 7–10. https://CRAN.R-project.org/doc/Rnews/ First citation in articleGoogle Scholar

  • Ziegler, M. (2011). Applicant faking: A look into the black box. The Industrial-Organizational Psychologist, 49(1), 29–36. First citation in articleGoogle Scholar

  • Ziegler, M., MacCann, C., & Roberts, R. D. (Eds.). (2012). Faking: Knowns, unknowns, and points of contention. New perspectives on faking in personality assessment (pp. 3–16). Oxford University Press. 10.1093/acprof:oso/9780195387476.001.0001 First citation in articleCrossref, Google Scholar