Does Response Scale Format Affect the Answering of Personality Scales?
Assessing the Big Five Dimensions of Personality with Different Response Scales in a Dependent Sample
Abstract
When developing a questionnaire, one puts much effort into item formulation. Whether the format of the corresponding response scales affects response behavior, however, has rarely been studied, to date. The present study investigates (1) the effects of the response scale direction (ranging from positive to negative vs. negative to positive) and (2) the match between numerical labeling and scale direction, i.e., assigning high numbers to the positive pole and low numbers to the negative pole or vice versa. These response scale effects were studied based on responses in the BFI-10, the short-scale version of the widely-used Big Five Inventory (BFI), assessing the Big Five dimensions of personality by two items each. Using a dependent sample design, subjects answered the 10 items using end-point labeled response scales ranging from the negative (labeled “1”) to the positive pole (labeled “8”) at Time 1. At Time 2 (approximately 3 weeks later), respondents were split into two conditions: The BFI-10 was administered again with an 8-point scale ranging from the positive to the negative pole. In the first condition, the positive pole was labeled “8” and the negative pole was labeled “1”; in the second condition, the positive pole was labeled “1” and the negative pole was labeled “8.” Results clearly support the notion that the direction of the response scale (Condition 1) does not affect response behavior. There were no differences in means, standard deviations, or in the intercorrelations patterns. However, there seems to be an intuitive match between the positive pole and high numerical labeling. When poles were counterintuitively labeled (Condition 2), significant differences could be identified for all analyses conducted.
References
Allgemeine Bevölkerungsumfrage der Sozialwissenschaften ALLBUS, 1998: Codebook (ZA-Nr. 3000). [
German General Social Survey ALLBUS, 1998: Codebook (ZA-No. 3000) ]. Cologne: Zentralarchiv für empirische Sozialforschung.(2001). The European Value Study: A third wave. Source book of the 1999/2000 European Value Study surveys. Tilburg, NL: WORC, Tilburg University.
(2004). International Social Survey Programme. Retrieved in April 2006 from www.issp.org/Documents/issp2004.pdf
(1991). The Big Five Inventory – Versions 4a and 54. Berkeley, CA: University of California, Berkeley, Institute of Personality and Social Research.
(1995). Behavioral frequency judgments: An accessibility-diagnosticity framework. Journal of Consumer Research, 22, 212–228.
(1998). Middle alternatives, acquiescence, and the quality of questionnaire data. Unpublished NORC report
(1989). Bias in quantifying judgments. Hillsdale, NJ: Erlbaum.
(1997). Die deutsche Version des Big Five Inventory (BFI): Übersetzung und Validierung eines Fragebogens zur Erfassung des Fünf-Faktoren-Modells der Persönlichkeit. [
The German version of the Big Five Inventory (BFI): Translation and validation of a questionnaire assessing the Five Factor Model of personality ]. Unpublished diploma thesis, University of Bielefeld, Germany(
in press ). Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German. Journal of Research in Personality.(2006). Die grossen Fünf: Nomological net and cultural differences in the U.S. and Germany. Manuscript in preparation
(1997). The validity and reliability of survey questions: A meta-analysis of MTMM studies. Sociological Methods and Research, 25, 341–383.
(1981). Questions and answers in attitude survey: Experiments on question form, wording, and context. New York: Academic Press.
(1995). The numeric values of rating scales: A comparison of their impact in mail surveys and telephone interviews. International Journal of Public Opinion Research, 7, 72–74.
(1991). Response alternatives: The impact of their choice and presentation order. In , Measurement errors in surveysNew York: Wiley
(1988). Judgments of relationship satisfaction: Inter- and intraindividual comparisons as a function of questionnaire structure. European Journal of Social Psychology, 18, 485–496.
(1992). Selbstberichtete Verhaltens- und Symptomhäufigkeiten: Was Befragte aus Antwortvorgaben des Fragebogens lernen. [Frequency reports of psychosomatic symptoms: What respondents learn from response alternatives.] Zeitschrift für Klinische Psychologie, 22, 197–208.
(1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93–105.
(1998). Formal features of rating scales and the interpretation of meaning. International Journal of Public Opinion Research, 10, 177–183.
(1985). Response scales: Effects of category range on reported behavior and subsequent judgments. Public Opinion Quarterly, 49, 388–395.
(1991). Rating scales: Numeric values may change the meaning of scale labels. Public Opinion Quarterly, 55, 570–582.
(1988). The range of response alternatives may determine the meaning of the question: Further evidence on informative functions of response alternatives. Social Cognition, 6, 107–117.
(2000). Handbook of parametric and nonparametric statistical procedures. Boca Raton, FL: Chapman & Hall/CRC
(1996). Asking sensitive questions: The impact of data collection, question format, and question context. Public Opinion Quarterly, 60, 275–304.
