Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Multistudy Report

The Effects of Questionnaire Length and Behavioral Consequences on Careless Responding

Published Online:July 02, 2019https://doi.org/10.1027/1015-5759/a000526

Abstract

Abstract. The current paper reports the results of two randomized experiments designed to test the effects of questionnaire length on careless responding (CR). Both experiments also examined whether the presence of a behavioral consequence (i.e., a reward or a punishment) designed to encourage careful responding buffers the effects of questionnaire length on CR. Collectively, our two studies found (a) some support for the main effect of questionnaire length, (b) consistent support for the main effect of the consequence manipulations, and (c) very limited support for the buffering effect of the consequence manipulations. Because the advancement of many subfields of psychology rests on the availability of high-quality self-report data, further research should examine the causes and prevention of CR.

References

  • Bäckström, M., & Björklund, F. (2019). Is reliability compromised towards the end of long personality inventories? European Journal of Psychological Assessment, 35, 14–23. https://doi.org/10.1027/1015-5759/a000363 First citation in articleLink, Google Scholar

  • Barnes, T. D. (2015). RStudio (Version 0.98.1103) [Function in RStudio]. Wright State University: Tyler Barnes. First citation in articleGoogle Scholar

  • Ben-Porath, Y. S., & Tellegen, A. (2008). MMPI-2-RF, Minnesota Multiphasic Personality Inventory-2 Restructured Form: Manual for administration, scoring and interpretation. Minneapolis, MN: University of Minnesota Press. First citation in articleGoogle Scholar

  • Berry, D. T., Wetter, M. W., Baer, R. A., Larsen, L., Clark, C., & Monroe, K. (1992). MMPI-2 random responding indices: Validation using a self-report methodology. Psychological Assessment, 4, 340–345. https://doi.org/10.1037/1040-3590.4.3.340 First citation in articleCrossref, Google Scholar

  • Bowling, N. A., Huang, J. L., Bragg, C. B., Khazon, S., Liu, M., & Blackmore, C. E. (2016). Who cares and who is careless? Insufficient effort responding as a reflection of respondent personality. Journal of Personality and Social Psychology, 111, 218–229. https://doi.org/10.1037/pspp0000085 First citation in articleCrossref, Google Scholar

  • Costa, P. T., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources. First citation in articleGoogle Scholar

  • Credé, M. (2010). Random responding as a threat to the validity of effect size estimates in correlational research. Educational and Psychological Measurement, 70, 596–612. First citation in articleCrossref, Google Scholar

  • Curran, P. G. (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4–19. First citation in articleCrossref, Google Scholar

  • DeSimone, J. A., Harms, P. D., & DeSimone, A. J. (2015). Best practice recommendations for data screening. Journal of Organizational Behavior, 36, 171–181. First citation in articleCrossref, Google Scholar

  • Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. First citation in articleCrossref, Google Scholar

  • Fisher, C. D. (1993). Boredom at work: A neglected concept. Human Relations, 46, 395–417. First citation in articleCrossref, Google Scholar

  • Galesic, M. (2006). Dropouts on the web: Effects of interest and burden experienced during an online survey. Journal of Official Statistics, 22, 313–328. First citation in articleGoogle Scholar

  • Galesic, M., & Bosnjak, M. (2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly, 73, 349–360. First citation in articleCrossref, Google Scholar

  • Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In I. MervieldeI. DearyF. De FruytF. OstendorfEds., Personality psychology in Europe (Vol. 7, pp. 7–28). Tilburg, The Netherlands: Tilburg University Press. First citation in articleGoogle Scholar

  • Gough, H. G., & Bradley, P. (1996). California Psychological Inventory: Administrator’s guide (3rd ed.). Palo Alto, CA: Consulting Psychologists Press. First citation in articleGoogle Scholar

  • Hagger, M. S., Wood, C., Stiff, C., & Chatzisarantis, N. L. (2010). Ego depletion and the strength model of self-control: A meta-analysis. Psychological Bulletin, 136, 495–525. First citation in articleCrossref, Google Scholar

  • Helm, S. (2005). Designing a formative measure for corporate reputation. Corporate Reputation Review, 8, 95–109. First citation in articleCrossref, Google Scholar

  • Herzog, A. R., & Bachman, J. G. (1981). Effects of questionnaire length on response quality. Public Opinion Quarterly, 45, 549–559. First citation in articleCrossref, Google Scholar

  • Hill, A. B., & Perkins, R. E. (1985). Towards a model of boredom. British Journal of Psychology, 76, 235–240. First citation in articleCrossref, Google Scholar

  • Holding, D. H. (1983). Fatigue. In G. R. J. HockeyEd., Stress and fatigue in human performance. Chichester, UK: Wiley. First citation in articleGoogle Scholar

  • Huang, J. L., Bowling, N. A., Liu, M., & Li, Y. (2015). Detecting insufficient effort responding with an infrequency scale: Evaluating validity and participant reactions. Journal of Business and Psychology, 30, 299–311. First citation in articleCrossref, Google Scholar

  • Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012). Detecting and deterring insufficient effort respond to surveys. Journal of Business and Psychology, 27, 99–114. First citation in articleCrossref, Google Scholar

  • Huang, J. L., Liu, M., & Bowling, N. A. (2015). Insufficient effort responding: Examining an insidious confound in survey data. Journal of Applied Psychology, 100, 828–845. First citation in articleCrossref, Google Scholar

  • Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213–236. First citation in articleCrossref, Google Scholar

  • Krosnick, J. A., Holbrook, A. L., Berent, M. K., Carson, R. T., Hanemann, W. M., Kopp, R. J., … Conaway, M. (2002). The impact of “no opinion” response options on data quality: Non-attitude reduction or an invitation to satisfice? Public Opinion Quarterly, 66, 371–403. First citation in articleCrossref, Google Scholar

  • Maniaci, M. R., & Rogge, R. D. (2014). Caring about carelessness: Participant inattention and its effects on research. Journal of Research in Personality, 48, 61–83. https://doi.org/10.1016/j.jrp.2013.09.008 First citation in articleCrossref, Google Scholar

  • Meade, A.W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17, 437–455. https://doi.org/10.1037/a0028085 First citation in articleCrossref, Google Scholar

  • Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. Boston, MA: Allyn & Bacon. First citation in articleGoogle Scholar

  • Teo, T. S., Lim, V. K., & Lai, R. Y. (1999). Intrinsic and extrinsic motivation in Internet usage. Omega, 27, 25–37. First citation in articleCrossref, Google Scholar

  • Widaman, K. F., Helm, J. L., Castro-Schilo, L., Pluess, M., Stallings, M. C., & Belsky, J. (2012). Distinguishing ordinal and disordinal interactions. Psychological Methods, 17, 615–622. https://doi.org/10.1037/a0030003 First citation in articleCrossref, Google Scholar