Do Researchers Anchor Their Beliefs on the Outcome of an Initial Study?
Testing the Time-Reversal Heuristic
Abstract
Abstract. As a research field expands, scientists have to update their knowledge and integrate the outcomes of a sequence of studies. However, such integrative judgments are generally known to fall victim to a primacy bias where people anchor their judgments on the initial information. In this preregistered study we tested the hypothesis that people anchor on the outcome of a small initial study, reducing the impact of a larger subsequent study that contradicts the initial result. Contrary to our expectation, undergraduates and academics displayed a recency bias, anchoring their judgment on the research outcome presented last. This recency bias is due to the fact that unsuccessful replications decreased trust in an effect more than did unsuccessful initial experiments. We recommend the time-reversal heuristic to account for temporal order effects during integration of research results.
References
1850). Novum organum, or true suggestions for the interpretation of nature (
(W. Wood , Trans.). London, UK: W. Pickering. Retrieved from https://books.google.nl/books?id=HPFoAAAAcAAJ2007). Thinking and deciding (4th ed.). New York, NY: Cambridge University Press.
(1967). Genesis of popular but erroneous psychodiagnostic observations. Journal of Abnormal Psychology, 72, 193–204.
(1969). Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. Journal of Abnormal Psychology, 74, 271–280.
(2016). Hidden multiplicity in exploratory multiway ANOVA: Prevalence and remedies. Psychonomic Bulletin & Review, 23, 640–647.
(2004). Bayesian data analysis (2nd ed.), Boca Raton, FL: Chapman & Hall/CRC.
(1995). Avoiding model selection in Bayesian social research. Sociological Methodology, 25, 165–173.
(1985). Counterfactual reasoning and accuracy in predicting personal events. Journal of Experimental Psychology: Learning, Memory and Cognition, 11, 719–731.
(2017). JASP. (Version 0.8.3.1) [Computer software]. Retrieved from https://jasp-stats.org/
. (1961). Theory of probability. Oxford, UK: Oxford University Press.
(1980). Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory, 6, 107–118.
(1980). The development of scientific thinking skills. New York, NY: Academic Press.
(1971). Order bias, the ideal rating, and the semantic differential. Journal of Marketing Research, 8, 375–378.
(2005). Confirmatory bias and confirmatory distortion. Journal of Child Custody: Research, Issues, and Practices, 2, 31–48.
(2017). BayesFactor package for R https://cran.r-project.org/web/packages/BayesFactor/index.html
(1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2, 175–220.
(2015). Estimating the reproducibility of psychological science. Science, 349, 1–8. https://doi.org/10.1126/science.aac4716
. (2012). Editors’ introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7, 528–530. https://doi.org/10.1177/1745691612465253
(2017). yarrr: A Companion to the e-Book “YaRrr!: The Pirate’s Guide to R”. (R package version 0.1.5) [Computer software manual]. Retrieved from https://CRAN.R-project.org/package=yarrr
(2017). R: A language and environment for statistical computing. Vienna, Austria: [Computer software manual]. Retrieved from http://www.R-project.org/
. (2009). Bayesian t-tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16, 225–237.
(2016). BFDA: Bayes factor design analysis package for R, https://github.com/nicebread/BFDA
(2016). Bayes factor design analysis: Planning for compelling evidence. Retrieved from https://ssrn.com/abstract=2722435 or https://doi.org/10.2139/ssrn.2722435
(2016). The pipeline project: Pre-publication independent replications of a single laboratory’s research pipeline. Journal of Experimental Social Psychology, 66, 55–67. https://doi.org/10.1016/j.jesp.2015.10.001
(2015). How do experts update beliefs? Lessons from a non-market environment. Journal of Behavioral and Experimental Economics, 57, 55–63.
(1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131.
(1960). On the failure to eliminate hypotheses in a conceptual task. The Quarterly Journal of Experimental Psychology, 12, 129–140.
(2009). How to quantify support for and against the null hypothesis: A flexible WinBUGS implementation of a default Bayesian t-test. Psychonomic Bulletin & Review, 16, 752–760.
(2012). Replication studies: Bad copy. Nature, 485, 298–300. https://doi.org/10.1038/485298a
(