Realism of Confidence Judgments
Abstract
This paper addresses measurement and conceptual issues related to the realism of people’s confidence judgments about their own cognitive abilities. We employed three cognitive tests: listening and reading subtests from the Test of English as a Foreign Language (TOEFL iBT) and a synonyms vocabulary test. The sample consisted of community college students. Our results show that the participants tend to be overconfident about their cognitive abilities on most tasks, representing poor realism. Significant group differences were noted with respect to gender and race/ethnicity: female and European American participants showed smaller levels of overconfidence than males and African Americans or Hispanics. We point out that there appear to be significant individual differences in the understanding of subjective probabilities, and these differences can influence the realism of confidence judgments.
References
(2006). Skilled or unskilled, but still unaware of it: How perceptions of difficulty drive miscalibration in relative comparisons. Journal of Personality and Social Psychology, 90, 60–77.
(1963). Manual and kit of reference tests for cognitive factors. Princeton, NJ: Educational Testing Service.
(1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
(2003). Bias score – An instance of systematic irrationality? In , Proceedings of Joint International Conference on Cognitive Science (Vol. 1, pp. 311–317). Sydney, Australia: University of New South Wales.
(1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, 1121–1134.
(1977). Do those who know more also know more about how much they know?. Organizational Behavior and Human Performance, 20, 159–183.
(1999). PARSCALE. Chicago, IL: Scientific Software.
(2008). An application of Rasch modeling to test-taker confidence and accuracy. Manuscript in preparation.
(1993). The psychology of judgment and decision making. New York: McGraw-Hill.
(1998). Calibration curves, scatterplots, and the distinction between general knowledge and perceptual tests. Learning and Individual Differences, 8, 28–51.
(1996). Confidence judgments in studies of individual differences. Personality and Individual Differences, 21, 971–986.
(2007). Confidence and cognitive test performance. Educational Psychology, 100, 961–976.
