Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Is Reliability Compromised Towards the End of Long Personality Inventories?

Published Online:November 07, 2016https://doi.org/10.1027/1015-5759/a000363

Abstract

Abstract. During very long self-rating sessions there is a risk that respondents will be tired and/or lose interest. Is this a concern for users of long personality inventories, such that the reliability becomes threatened in the latter half when respondents have made hundreds of personality self-ratings? Two thousand three hundred fifty-two volunteers completed long (≈ 500 items) personality inventories on the Internet, where items were presented in a unique random order for each participant. Perhaps counterintuitively, there was no evidence that reliability is threatened as respondents approach the end of a long personality inventory. If anything, the ratings in the second half of the inventories had higher reliability than ratings in the first half. Ratings were quicker towards the end of the inventories, but equally reliable. The criterion validity, estimated using Paunonen’s Behavior Report Form, was maintained too. The current results provide little reason to mistrust responses to items that appear towards the end of long personality inventories.

References

  • Bäckström, M., Björklund, F. & Larsson, M. R. (2009). Five-factor inventories have a major general factor related to social desirability which can be reduced by framing items neutrally. Journal of Research in Personality, 43, 335–344. doi: 10.1016/j.jrp.2008.12.013 First citation in articleCrossref, Google Scholar

  • Bäckström, M., Björklund, F. & Larsson, M. R. (2014). Criterion validity is maintained when items are evaluatively neutralized: Evidence from a full‐scale five‐factor model inventory. European Journal of Personality, 28, 620–633. doi: 10.1002/per.1960 First citation in articleCrossref, Google Scholar

  • Bendig, A. W. (1955). Rater reliability and “judgmental fatigue”. Journal of Applied Psychology, 39, 451–454. doi: 10.1037/h0046015 First citation in articleCrossref, Google Scholar

  • Berry, D. R., Wetter, M. W., Baer, R. A., Larsen, L., Clark, C. & Monroe, K. (1992). MMPI-2 random responding indices: Validation using a self-report methodology. Psychological Assessment, 4, 340–345. doi: 10.1037/1040-3590.4.3.340 First citation in articleCrossref, Google Scholar

  • Burisch, M. (1984). Approaches to personality inventory construction: A comparison of merits. American Psychologist, 39, 214–227. doi: 10.1037/0003-066X.39.3.214 First citation in articleCrossref, Google Scholar

  • Costa, P. T. & McCrae, R. R. (1992). Revised NEO personality inventory (NEO-PI-R) and NEO five-factor inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources. First citation in articleGoogle Scholar

  • Credé, M., Harms, P., Niehorster, S. & Gaye-Valentine, A. (2012). An evaluation of the consequences of using short measures of the Big Five personality traits. Journal of Personality and Social Psychology, 102, 874–888. doi: 10.1037/a0027403 First citation in articleCrossref, Google Scholar

  • Galesic, M. & Bosnjak, M. (2009). Effects of questionnaire length on participation and indicators of response quality in a Web survey. The Public Opinion Quarterly, 73, 349–360. doi: 10.1093/poq/nfp031 First citation in articleCrossref, Google Scholar

  • Goldberg, L. R. (1990). An alternative “description of personality”: The Big Five factor structure. Journal of Personality and Social Psychology, 59, 1216–1229. doi: 10.1037/0022-3514.59.6.1216 First citation in articleCrossref, Google Scholar

  • Hofstee, W. K., de Raad, B. & Goldberg, L. R. (1992). Integration of the Big Five and circumplex approaches to trait structure. Journal of Personality and Social Psychology, 63, 146–163. doi: 10.1037/0022-3514.63.1.146 First citation in articleCrossref, Google Scholar

  • Johnson, J. A. (2005). Ascertaining the validity of individual protocols from Web-based personality inventories. Journal of Research in Personality, 39, 103–129. doi: 10.1016/j.jrp.2004.09.009 First citation in articleCrossref, Google Scholar

  • Knowles, E. (1988). Item context effects on personality scales: Measuring changes the measure. Journal of Personality and Social Psychology, 55, 312–320. doi: 10.1037/0022-3514.55.2.312 First citation in articleCrossref, Google Scholar

  • Kurtz, J. E. & Parrish, C. L. (2001). Semantic response consistency and protocol validity in structured personality assessment: The case of the NEO-PI-R. Journal of Personality Assessment, 76, 315–332. doi: 10.1207/S15327752JPA7602_12 First citation in articleCrossref, Google Scholar

  • McCrae, R. R. & Costa, T. P.Jr. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52, 81–90. doi: 10.1037/0022-3514.52.1.81 First citation in articleCrossref, Google Scholar

  • McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Paunonen, S. V. (2003). Big five factors of personality and replicated predictions of behavior. Journal of Personality and Social Psychology, 84, 411–422. doi: 10.1037/0022-3514.84.2.411 First citation in articleCrossref, Google Scholar

  • Revelle, W. (2015). psych: Procedures for personality and psychological research. Evanston, IL: Northwestern University. http://CRAN.R-project.org/package=psych Version=1.5.1 First citation in articleGoogle Scholar

  • Revelle, W. & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrica, 74, 145–154. doi: 10.1007/S11336-008-9102-Z First citation in articleCrossref, Google Scholar

  • Robins, R. W., Hendin, H. M. & Trzesniewski, K. H. (2001). Measuring global self-esteem: Construct validation of a single-item measure and the Rosenberg Self-Esteem Scale. Personality and Social Psychology Bulletin, 27, 151–161. doi: 10.1177/0146167201272002 First citation in articleCrossref, Google Scholar

  • Rolstad, S., Adler, J. & Rydén, A. (2011). Response burden and questionnaire length: Is shorter better? A review and meta-analysis. Value in Health, 14, 1101–1108. doi: 10.1016/j.jval.2011.06.003 First citation in articleCrossref, Google Scholar