A Meta-Analysis of Judgment Achievement as Defined by the Lens Model Equation
Abstract
This research determined the extent of judgment achievement (i.e., accuracy) across different decision-making domains (e.g., medicine, business, education, psychology). Judgment achievement was examined in terms of the lens model equation (Tucker, 1964). A meta-analysis of 29 studies incorporating 1,032 people across 43 judgment tasks was performed. Overall judgment achievement across different tasks was found to be moderate (r = .42), ranging from .22 for studies in the area of psychology to .58 for those in other professional areas. Methodological moderator factors (type of correlation, database) used in task analysis or the number of cues used in tasks were also considered. The findings indicated that research on judgment achievement should examine judgment tasks relating to specific research areas.
References marked with an asterisk (*) indicate studies included in the meta-analysis (see Table 1).
References
(2001). Judgmental bootstrapping: Inferring experts’ rules for forecasting. In , Principles of forecasting (pp. 171–192). Philadelphia, PA: Springer.
(1982). An empirical study of budget-related predictions of corporate executives. Journal of Accounting Research, 20, 440–449.
(2000). A review and analysis of research on the test-retest reliability of professional judgment. Journal of Behavioral Decision Making, 13, 277–294.
(2001). Judgment of factors influencing interest: An Australian study. Journal of Vocational Education Research, 26, 1–13.
(1989). Effects of cognitive feedback on performance. Psychological Bulletin, 106, 410–433.
(1996). Dyad rapport and the accuracy of its judgment across situations: A lens model analysis. Journal of Personality and Social Psychology, 71, 110–129.
(1988). What have we learned about human judgment from 30 years of policy capturing. In , Human judgment: The SJT view (pp. 75–114). Amsterdam: North Holland.
(1976). Note on clinical judgment and the formal characteristics of clinical tasks. Psychological Bulletin, 83, 778–782.
(1952). The conceptual framework of psychology. International encyclopedia of unified science. Chicago: University of Chicago Press.
(1955). Representative design and probabilistic theory in a functional psychology. Psychological Review, 62, 193–216.
(1981). General conditions for the success of bootstrapping models. Organizational Behavior and Human Performance, 27, 411–422.
(1996). Judgment analysis: Theory, methods, and applications. New York: Academic Press.
(1986). Teachers’ predictions of children’s early reading achievement: An application of social judgment theory. American Educational Research Journal, 23, 41–64.
(1990). Predicting violence in newly admitted inmates: A lens model analysis of staff decision making. Criminal Justice and Behavior, 17, 431–447.
(1996). Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty. Cognition, 58, 1–73.
(2004). The role of representative design in an ecological approach to cognition. Psychological Bulletin, 130, 959–988.
(1974). Cue definition and residual judgment. Organizational Behavior and Human Performance, 12, 30–49.
(1985). Nonverbal cues in the employment interview: Links between applicant qualities and interviewer judgments. Journal of Applied Psychology, 70, 729–736.
(1991). How to make cognitive illusions disappear: Beyond heuristics and biases. In , European review of social psychology (Vol. 2, pp. 83–115). New York: Wiley.
(1998). Ecological intelligence: An adaptation for frequencies. In , The evolution of mind (pp. 9–29). New York: Oxford University Press.
(1996). Reasoning the fast and frugal way: Models of bounded rationality. Psychological Review, 103, 650–669.
(1999). Simple heuristics that make us smart. Oxford: Oxford University Press.
(1976). Man versus model of man: Just how conflicting is that evidence? Organizational Behavior and Human Performance, 16, 13–22.
(1978). Can we learn anything about interviewing real people from “interviews” of paper people? Two studies of the external validity of a paradigm. Organizational Behavior and Human Performance, 22, 165–192.
(2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12, 19–30.
(1964). Analyzing the components of clinical inference. Psychological Review, 71, 438–456.
(1975). Social judgment theory. In , Human judgment and decision processes (pp. 271–317). New York: Academic Press, Inc.
(1995). Why are judgments less consistent in less predictable task situations? Organizational Behavior and Human Decision Processes, 63, 247–263.
(2004). Effects of judges’ forecasting on their later combination for forecasts for the same outcomes. International Journal of Forecasting, 20, 391–409.
(1999). An annotated bibliography of all published cue probability learning studies. Retrieved March 21, 2005 from www.brunswik.org/resources/mcplbib.doc.
(2004). Methods of meta-analysis: Correcting error and bias in research findings (Vol. 2). Newbury Park, CA: Sage.
(1982). Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage.
(1964). Some methodological considerations in multiple-cue probability learning studies. Psychological Review, 71, 42–60.
(1982). Judgment under uncertainty: Heuristics and biases. London: Cambridge University Press.
(2008). Determinants of linear judgment: A meta-analysis of lens studies. Psychological Bulletin, 134, 404–426.
(2007). The idiographic approach in social judgment theory: A review of components of the lens model equation components. International Journal of Idiographic Science, 2.
(1988). An exploratory study of physicians’ clinical judgment: An application of social judgment theory. Evaluation and the Health Professions, 11, 178–200.
(1992). How quantity judgment changes as the number of cues increases: An analytical framework and review. Psychological Bulletin, 112, 363–377.
(1989). Expert systems should be more accurate than human experts: Evaluation procedures from human judgment and decision making. IEEE Transactions on Systems, Man, and Cybernetics, 19, 647–657.
(1986). Graphic representation of judgmental information. Human-Computer Interaction, 2, 179–200.
(1984). What makes the Muller a liar: A multiple-cue approach. Perception and Psychophysics, 36, 234–244.
(1987). Assessing the accuracy of financial analyst security return predictions. Accounting Organizations and Society, 12, 331–340.
(1975). Integration of information in a clinical judgment task, an empirical comparison of six models. Perceptual and Motor Skills, 40, 343–356.
(1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157–159.
(2001). The sounds and sights of intelligence: A lens model channel analysis. Personality and Social Psychology Bulletin, 27, 187–200.
(1976). Judgment theory applied to the selection of life insurance salesmen. Organizational Behavior and Human Performance, 16, 231–249.
(1993). Metaanalyse des Zusammenhangs von Interesse und schulischer Leistung. [
A meta-analysis concerning the relation between interest and achievement in school ]. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 25, 120–148.(1992). How much information does an expert use? Is it relevant? Acta Psychologica, 81, 75–86.
(2002). Domain differences in expertise. (Working paper). Kansas State University, KS: Manhattan.
(1990). Relative evaluation of subjective and objective measures of expectations formation. Quarterly Review of Economics and Business, 30, 64–74.
(2003). Factors influencing prescribing decisions in the treatment of depression: A social judgment theory approach. Applied Cognitive Psychology, 17, 51–63.
(1989). Lens model analysis of hemodynamic status in the critically ill. Medical Decision Making, 9, 243–261.
(1972). A lens model analysis of a bookbag and poker chip experiment: A methodological note. Organizational Behavior and Human Performance, 8, 450–455.
(1990). Notes and correspondence: A decomposition of the correlation coefficient and its use in analyzing forecasting skill. American Meteorological Society, 5, 661–666.
(1997). The importance of the task in analyzing expert judgment. Organizational Behavior and Human Decision Processes, 69, 205–219.
(1981). Statistical versus clinical lie detection. American Psychologist, 36, 488–496.
(1993). The relation between interest congruence and satisfaction: A metaanalysis. Journal of Vocational Behavior, 42, 253–264.
(1964). A suggested alternative formulation in the developments by Hursch, Hammond, and Hursch and by Hammond, Hursch, and Todd. Psychological Review, 71, 528–530.
(1989). Social workers’ decision making about the violent client. Social Work Research and Abstracts, 25, 17–20.
(1983). Reliability, accuracy, and decision-making strategy in clinical predictions of imminent dangerousness. Journal of Consulting and Clinical Psychology, 51, 815–825.
(1971). Man versus model of man revisited: The forecasting of graduate school success. Journal of Personality and Social Psychology, 19, 100–106.
(1985). Evaluationsforschung: Aufgaben, Probleme und Anwendungen. [
Evaluation research: Tasks, problems and applications ]. Berlin: Springer.(2005). Meta-analysis of correlation coefficients (Schmidt-Hunter method). Retrieved January 7, 2006 from www.sussex.ac.uk/Users/danw/ESM/SHcorrmeta.SPS.
(1979). Properties of judgment models in a financial setting. Organizational Behavior and Human Performance, 23, 73–85.
