The Tendency to Fall for Distracting Information While Making Judgments
Development and Validation of the Objective Heuristic Thinking Test
Abstract
Despite much research on thinking biases such as the representativeness, availability, and anchoring heuristics, a psychometrically sound measurement instrument for assessing the degree of heuristic thinking is still missing. Therefore, it was the goal of this study to develop and validate a new test to assess the degree of heuristic thinking associated with three particular thinking heuristics (i.e., the representativeness, availability, and anchoring heuristics). The resulting Objective Heuristic Thinking Test (OHTT) was evaluated with regard to its internal consistency, factor structure, construct validity, and stability in an internet sample (N = 300) and an independent laboratory sample (N = 55). Exploratory factor analyses resulted in three latent factors that represented the three OHTT subscales (i.e., representativeness, availability, and anchoring factors). Results revealed a low to sufficient internal consistency for each of the three scales. Further analyses indicated convergent correlations of the OHTT scales with related constructs such as field-independency. Furthermore, good stability of the test scores was shown. Conclusions are drawn regarding possible future applications of the OHTT as a promising tool for studying the origins of heuristic thinking processes.
References
(1991). Development of judgmental heuristics and logical reasoning: Training counteracts the representativeness heuristic. Cognitive Development, 6, 195–217. doi: 10.1016/0885-2014(91)90036-D
(1980). Verbaler Kurzintelligenztest
[Verbal Short Intelligence Test] . Göttingen, Germany: Hogrefe.(1994). Sensation seeking: A new conceptualization and a new scale. Personality and Individual Differences, 16, 289–296. doi: 10.1016/0191-8869(94)90165-1
(1998). A model of investor sentiment. Journal of Financial Economics, 49, 307–343. doi: 10.1016/S0304-405X(98)00027-0
(2008). IST – Intelligence structure test. Göttingen, Germany: Hogrefe.
(2007). Trait anxiety and reasoning under uncertainty. Personality and Individual Differences, 43, 827–838. doi: 10.1016/j.paid.2007.02.007
(1980). Administrators’ estimations of the probability of outcomes of school desegregation: A field test of the availability heuristic. Organizational Behavior and Human Performance, 26, 97–114. doi: 10.1016/0030-5073(80)90049-5
(2013). Representativeness heuristic, investor sentiment and overreaction to accounting earnings: The case of the Tunisian stock market. Procedia – Social and Behavioral Sciences, 81, 9–21. doi: 10.1016/j.sbspro.2013.06.380
(2003). Superstitious responding and frequency matching in the positive bias and gamblers fallacy effects. Organizational Behavior and Human Decision Processes, 91, 119–127. doi: 10.1016/S0749-5978(03)00064-5
(2003). The representativeness heuristic: Influence on nurses’ decision making. Applied Nursing Research, 16, 201–204. doi: 10.1016/S0897-1897(03)00043-0
(2002). The fragile basic anchoring effect. Journal of Behavioral Decision Making, 15, 65–77. doi: 10.1002/bdm.403
(1999). The effects of financial incentives in experiments: A review and capital-labor-production framework. Journal of Risk and Uncertainty, 19, 7–42. doi: 10.1023/A:1007850605129
(2013). A literature review of cognitive biases in negotiation processes. International Journal of Conflict Management, 24, 374–398. doi: 10.1108/IJCMA-08-2012-0064
(1978). The effect of imagining an event on expectations for the event: An interpretation in terms of the availability heuristic. Journal of Experimental Social Psychology, 14, 88–96. doi: 10.1016/0022-1031(78)90062-8
(1941). An objective test of character-temperament I. Journal of General Psychology, 25, 59–73.
(1944). An objective test of character-temperament: II. The Journal of Social Psychology, 19, 99–114. doi: 10.1080/00224545.1944.9918805
(1968). What is “objective” in “Objective Personality Tests”. In , Readings in psychological tests and measurements (pp. 163–168). Homewood, IL: Dorsey Press.
(1967). Objective Personality and Motivation Tests: A theoretical introduction and practical compendium. Chicago, IL: University of Illinois.
(1950). The objective measurement of dynamic traits. Educational and Psychological Measurement, 10, 223–247. doi: 10.1177/001316445001000204
(1980). Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of Personality and Social Psychology, 39, 752–766.
(1994). The limits of anchoring. Journal of Behavioral Decision Making, 7, 223–242. doi: 10.1002/bdm.3960070402
(1999). Motivated heuristic and systematic processing. Psychological Inquiry, 10, 44–49.
(2001). A study of the use of past experiences in clinical decision making in emergency situations. International Journal of Nursing Studies, 38, 591–599. doi: 10.1016/S0020-7489(00)00096-1
(2010). Cognitive therapy of anxiety disorders. New York, NY: Guilford Press.
(1993). Sensation seeking, gambling and gambling addictions. Addiction, 88, 541–554.
(1992). Disruption of the representativeness heuristic: Can we be perturbed into using correct probabilistic reasoning? Educational Studies in Mathematics, 23, 163–178. doi: 10.1007/BF00588054
(1985). Does the stock market overreact? The Journal of Finance, 40, 793–805. doi: 10.1111/j.1540-6261.1985.tb05004.x
(2006). The anchoring-and-adjustment heuristic: Why the adjustments are insufficient. Psychological Science, 17, 311–318. doi: 10.1111/j.1467-9280.2006.01704.x
(1980). Verbal reports as data. Psychological Review, 87, 215–251. doi: 10.1037/0033-295X.87.3.215
(1999). Protocol analysis: Verbal reports as data (3rd ed.). Cambridge, MA: MIT Press.
(2010). Biases in judgmental adjustments of statistical forecasts: The role of individual differences. International Journal of Forecasting, 26, 116–133. doi: 10.1016/j.ijforecast.2009.02.005
(2013). Field independence: Reviewing the evidence. British Journal of Educational Psychology, 83, 210–224. doi: 10.1111/bjep. 12015
(1982). Statistical judgment: A further test of the representativeness construct. Acta Psychologica, 51, 91–103. doi: 10.1016/0001-6918(82)90054-3
(2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25–42. doi: 10.1257/089533005775196732
(2008). Cognitive and behavioral therapies. In , Psychiatry (3rd ed.). (pp. 1920–1947). Oxford: Wiley.
(2011). A literature review of the anchoring effect. The Journal of Socio-Economics, 40, 35–42. doi: 10.1016/j.socec.2010.10.008
(1991). How to make cognitive illusions disappear: Beyond “heuristics and biases”. European Review of Social Psychology, 2, 83–115. doi: 10.1080/14792779143000033
(2011). Heuristic decision making. Annual Review of Psychology, 62, 451–482. doi: 10.1146/annurev-psych-120709-145346
(1996). Reasoning the fast and frugal way: Models of bounded rationality. Psychological Review, 103, 650–669. doi: 10.1037/0033-295X.103.4.650
(1985). The hot hand in basketball: On the misperception of random sequences. Cognitive Psychology, 17, 295–314. doi: 10.1016/0010-0285(85)90010-6
(2004). Faites vos jeux! Another look at sensation seeking and pathological gambling. Personality and Individual Differences, 37, 917–931. doi: 10.1016/j.paid.2003.10.014
(1999). Vorstellung und Validierung des Gestaltwahrnehmungstests zur Messung der Feldabhängigkeit
[Presentation and validation of the Gestaltwahrnehmungstest aimed at measureung field-independency] . Diagnostica, 45, 20–34. doi: 10.1026//0012-1924.45.1.20(1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179–185.
(2000). Konstruktion und Evaluation eines adaptiven Matrizentests
[Construction and evaluation of an adaptive matrices test] . Diagnostica, 46, 182–188. doi: 10.1026//0012-1924.46.4.182(1972). A factor analysis of selected markers for objective personality factors. Multivariate Behavioral Research, 7, 451–476.
. (2011). IBM SPSS Statistics for Windows, Version 20.0. Armonk, NY: IBM.
(2013). Car stickiness: Heuristics and biases in travel choice. Transport Policy, 25, 158–168. doi: 10.1016/j.tranpol.2012.11.004
(2010). Applied dimensionality and test structure assessment with the START-M mathematics test. International Journal of Educational and Psychological Assessment, 6, 104–125.
(1983). Representativeness in judgmental predictions of corporate bankruptcy. The Accounting Review, 58, 78–97.
(1972). Subjective probability: A judgment of representativeness. Cognitive Psychology, 3, 430–454. doi: 10.1016/0010-0285(72)90016-3
(1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263–291.
(1982). On the study of statistical intuitions. Cognition, 11, 123–141. doi: 10.1016/0010-0277(82)90022-1
(1994). The two fallacies of gamblers: Type I and type II. Organizational Behavior and Human Decision Processes, 60, 75–89.
(1983). A construct validation of the objective-analytic test battery. Personality and Individual Differences, 5, 323–337.
(2009). The technique of objective personality-tests sensu R. B. Cattell nowadays: The Viennese pool of computerized tests aimed at experiment-based assessment of behavior. Acta Psychologica Sinica, 41, 1024–1036.
(2011). Short assessment of the Big Five: Robust across survey methods except telephone interviewing. Behavior Research Methods, 43, 548–567. doi: 10.3758/s13428-011-0066-z
(1981). STAI – State-Trait-Angstinventar. Weinheim, Germany: Beltz.
(2002). Evaluation of a behavioral measure of risk taking: The Balloon Analogue Risk Task (BART). Journal of Experimental Psychology: Applied, 8, 75–84.
(2007). Searching for anchoring effects in a naturalistic environment: Evidence from the Hong Kong horserace betting market. Journal of Gambling Business and Economics, 1, 69–84.
(2006). FACTOR: A computer program to fit the exploratory factor analysis model. Behavioral Research Methods, Instruments and Computers, 38, 88–91.
(2005). The relationship between the five-factor model of personality and symptoms of clinical disorders: A meta-analysis. Journal of Psychopathology and Behavioral Assessment, 27, 101–114. doi: 10.1007/s10862-005-5384-y
(2007). Susceptibility to anchoring effects: How openness-to-experience influences responses to anchoring cues. Judgment and Decision Making, 2, 48–53.
(1998). Anchoring in the interpretation of probability expressions. Journal of Pragmatics, 30, 723–733. doi: 10.1016/S0378-2166(98)00011-3
(2005). Judgment and decision making. In , Handbook of cognition (pp. 321–338). London, UK: Sage.
(2010). Associative processes in intuitive judgment. Trends in Cognitive Sciences, 14, 435–440. doi: 10.1016/j.tics.2010.07.004
(2013). The relationship between gambling affinity, impulsivity, sensation seeking, superstition, and irrational beliefs: An empirical study among committed gamblers. Aloma, 31, 109–121.
(2001). The semantics of anchoring. Organizational Behavior and Human Decision Processes, 86, 234–255. doi: 10.1006/obhd.2001.2954
(2006). Belastbarkeits-Assessment: Computerisierte Objektive Persönlichkeits-Testbatterie – Deutsch (BAcO-D)
[Stress resistance assessment: Computerized objective test-battery – German version] . Frankfurt a. Main, Germany: Harcourt Assessment.(
in press ). Objective Personality Tests. In , Behavior-based assessment: Going beyond self-report in the personality, affective, motivation and social domains. Göttingen, Germany: Hogrefe.(1986). The elaboration likelihood model of persuasion. In , Advances in experimental social psychology (pp. 123–205). Orlando, FL: Academic Press.
(1999). The elaboration likelihood model: Current status and controversies. In , Dual-process theories in social psychology (pp. 37–72). New York, NY: Guilford Press.
(1989). Thinking the unthinkable: The effects of anchoring on likelihood estimates of nuclear war. Journal of Applied Social Psychology, 19, 67–91. doi: 10.1111/j.1559-1816.1989.tb01221.x
(1978). From individual to group impressions: Availability heuristics in stereotype formation. Journal of Experimental Social Psychology, 14, 237–255. doi: 10.1016/0022-1031(78)90013-6
(2003). Validation of the Arnett Inventory of Sensation Seeking (AISS): Efficiency to predict the willingness towards occupational chance, and affection by social desirability. Personality and Individual Differences, 35, 1307–1314. doi: 10.1016/S0191-8869(02)00351-3
(2002). Two systems of reasoning. In , Heuristics and biases. The psychology of intuitive judgment (pp. 379–396). Cambridge, UK: Cambridge Univ. Press.
(1970). Test Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press.
(1997). Reasoning independently of prior belief and individual differences in actively open-minded thinking. Journal of Educational Psychology, 89, 342–357. doi: 10.1037/0022-0663.89.2.342
(2008). On the relative independence of thinking biases and cognitive ability. Journal of Personality and Social Psychology, 94, 672–695. doi: 10.1037/0022-3514.94.4.672
(2008). Anchoring effects on performance judgments. Organizational Behavior and Human Decision Processes, 107, 29–40. doi: 10.1016/j.obhdp.2008.01.003
(2011). The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks. Memory & Cognition, 39, 1275–1289. doi: 10.3758/s13421-011-0104-1
(1973). Availability: A heuristic for judging frequency and probability. Cognitive Psychology, 5, 207–232. doi: 10.1016/0010-0285(73)90033-9
(1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131. doi: 10.1126/science.185.4157.1124
(1990). Shortening the State-Trait Anxiety Inventory. Journal of Clinical Epidemiology, 43, 995–1000.
(2010). Elaboration and numerical anchoring: Implications of attitude theories for consumer judgment and decision making. Journal of Consumer Psychology, 20, 5–16.
(2013). The Cognitive Reflection Test: How much more than Numerical Ability? In , Proceedings of the 35th Annual Conference of the Cognitive Science Society (pp. 1587–1593). Austin, TX: Cognitive Science Society Retrieved from http://mindmodeling.org/cogsci2013/papers/0296/paper0296.pdf
(1996). The emotional Stroop task and psychopathology. Psychological Bulletin, 120, 3–24. doi: 10.1037/0033-2909.120.1.3
(1977). Field-dependent and field-independent cognitive styles and their educational implications. Review of Educational Research, 47, 1–64. doi: 10.3102/00346543047001001
(1971). A manual for the embedded figures tests. Palo Alto, CA: Consulting Psychologists Press.
(1995). The significance of Brunswik-Symmetry for psychological research and assessment. European Journal of Psychological Assessment, 11, 59–60.
(2012). Anchoring bias in online voting. EPL (Europhysics Letters), 100, 1–5. doi: 10.1209/0295-5075/100/68002
(2005). Faites vos jeux anouveau: Still another look at sensation seeking and pathological gambling. Personality and Individual Differences, 39, 361–365.
