Reliability and Convergence of Conflict Effects
An Examination of Evidence for Domain-General Attentional Control
Abstract
Abstract. Recent work in attentional control has suggested that conflict effects measured across different tasks are not reliable and by extension unrelated. The lack of correlation between these conflict effects is in juxtaposition not only to theoretical predictions of a domain-general attentional control mechanism but also to a large body of individual differences research that has used these tasks to show evidence for an attentional control construct and its relatedness to other psychological constructs. In an effort to address this, we fit hierarchical models to each task that modeled trial-to-trial variability in response times to assess the extent to which the parameter estimates for the conflict effect correlated across tasks. We compared this method of assessing shared variance to more traditional summed difference score estimates of the conflict effect by analyzing data from a large-scale individual differences experiment, in which N = 582 subjects completed a Stroop, Flanker, and Simon task. Across tasks, we found that while the reliability of the conflict was sufficiently high and the between-task conflict effect significantly correlated, the magnitude of the between-task correlation was low. We discuss the implications of these results as providing more support for a domain-specific than domain-general attentional control mechanism.
References
1991). The adaptive nature of human categorization. Psychological Review, 98(3), 409–429. 10.1037/0033-295X.98.3.409
(2017). Cognitive control in schizophrenia: Psychological and neural mechanisms. In T. Egner (Ed.), The Wiley handbook of cognitive control (pp. 556–580). Wiley. 10.1002/9781118920497.ch31
(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. 10.18637/jss.v067.i01
(2001). Conflict monitoring and cognitive control. Psychological Review, 108(3), 624–652. 10.1037/0033-295X.108.3.624
(2012). In support of a distinction between voluntary and stimulus-driven control: A review of the literature on proportion congruent effects. Frontiers in Psychology, 3, 367. 10.3389/fpsyg.2012.00367
(2014). Distributional reaction time properties in the Eriksen task: Marked differences or hidden similarities with the Simon task? Psychonomic Bulletin & Review, 21(4), 1003–1010. 10.3758/s13423-013-0561-6
(2017). The neural basis of response inhibition and substance abuse. In T. Egner (Ed.), The Wiley handbook of cognitive control (pp. 581–601). Wiley. 10.1002/9781118920497.ch32
(1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. 10.1037/h0040957
(2017). Cognitive control in attention deficit hyperactivity disorder. In T. Egner (Ed.), The Wiley handbook of cognitive control (pp. 602–618). Wiley. 10.1002/9781118920497.ch33
(2008). Multiple conflict-driven control mechanisms in the human brain. Trends in Cognitive Sciences, 12(10), 374–380. 10.1016/j.tics.2008.07.001
(2014). Creatures of habit (and control): A multi-level learning perspective on the modulation of congruency effects. Frontiers in Psychology, 5, 1247. 10.3389/fpsyg.2014.01247
(1999). Individual differences in working memory capacity and what they tell us about controlled attention, general fluid intelligence, and functions of the prefrontal cortex. In A. Miyake (Ed.), Models of working memory: Mechanisms of active maintenance and executive control (pp. 102–134). Cambridge University Press. 10.1017/CBO9781139174909.007
(2019). Large-scale analysis of test\x{2013}retest reliabilities of self-regulation measures. Proceedings of the National Academy of Sciences, 116(12), 5472–5477. 10.1073/pnas.1818430116
(2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195–212. 10.3758/s13428-017-0862-1
(2018). A tutorial on regularized partial correlation networks. Psychological Methods, 24(4), 617–634.
(1974). Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception & Psychophysics, 16(1), 143–149. doi: 10.3758/BF03203267
(2016). An investigation of the reliability and self-regulatory correlates of conflict adaptation. Experimental Psychology, 63(4), 237–247. 10.1027/1618-3169/a000328
(2004). The relations among inhibition and interference control functions: A latent-variable analysis. Journal of Experimental Psychology: General, 133(1), 101–135. 10.1037/0096-3445.133.1.101
(2010). Analyzing the generality of conflict adaptation effects. Journal of Experimental Psychology: Human Perception and Performance, 36(1), 147–161. 10.1037/a0017598
(2016). Use of internal consistency coefficients for estimating reliability of experimental task scores. Psychonomic Bulletin & Review, 23(3), 750–763. 10.3758/s13423-015-0968-3
(2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50(3), 1166–1186. 10.3758/s13428-017-0935-1
(2016). Classic articles in psychology in the science citation index expanded: A bibliometric analysis. British Journal of Psychology, 107(4), 768–780. 10.1111/bjop.12163
(2003). Working-memory capacity and the control of attention: The contributions of goal neglect, response competition, and task set to Stroop interference. Journal of Experimental Psychology: General, 132(1), 47–70. 10.1037/0096-3445.132.1.47
(1992). Dimensional overlap and dimensional relevance in stimulus–response and stimulus–stimulus compatibility. North-Holland.
. (1994). The way irrelevant dimensions are processed depends on what they overlap with: The case of Stroop- and Simon-like stimuli. Psychological Research, 56(3), 130–135. 10.1007/BF00419699
. (1990). Dimensional overlap: Cognitive basis for stimulus–response compatibility – A model and taxonomy. Psychological Review, 97(2), 253–270.
(1958). The utilization of unreliable difference scores. ETS Research Bulletin Series, 1958(1), i-6. 10.1002/j.2333-8504.1958.tb00077.x
(2012). The nature and organization of individual differences in executive functions: Four general conclusions. Current Directions in Psychological Science, 21(1), 8–14. 10.1177/0963721411429458
(2016). The role of test–retest reliability in measuring individual and group differences in executive functioning. Journal of Neuroscience Methods, 274, 81–93. 10.1016/j.jneumeth.2016.10.002
(2010). Exploring the differences in distributional properties between Stroop and Simon effects using delta plots. Attention, Perception & Psychophysics, 72(7), 2013–2025. 10.3758/APP.72.7.2013
(2016). Cognitive predictors of a common multitasking ability: Contributions from working memory, attention control, and fluid intelligence. Journal of Experimental Psychology: General, 145(11), 1473–1492. 10.1037/xge0000219
(2018). Should we stop thinking about inhibition? Searching for individual and age differences in inhibition ability. Journal of Experimental Psychology: Learning, Memory, and Cognition, 44(4), 501–526. 10.1037/xlm0000450
(2019). A psychometrics of individual differences in experimental tasks. Psychonomic Bulletin & Review, 26(2), 452–467. 10.3758/s13423-018-1558-y
(1967). Auditory S–R compatibility: The effect of an irrelevant cue on information processing. Journal of Applied Psychology, 51(3), 300–304. 10.1037/h0020586
(1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662. 10.1037/h0054651
(2017). The importance of arousal for variation in working memory capacity and attention control: A latent variable pupillometry study. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43(12), 1962–1987. 10.1037/xlm0000421
(2010). Working memory capacity: Attention control, secondary memory, or both? A direct test of the dual-component model. Journal of Memory and Language, 62(4), 392–406. 10.1016/j.jml.2010.02.001
(2009). Examining the relations among working memory capacity, attention control, and fluid intelligence from a dual-component framework. Psychological Test and Assessment Modeling, 51(4), 388–402.
(2018). Are cognitive control processes reliable? Journal of Experimental Psychology: Learning, Memory, and Cognition, 45(5), 765–778. 10.1037/xlm0000632
(1999). A parallel distributed processing model of stimulus\x{2013}stimulus and stimulus\x{2013}response compatibility. Cognitive Psychology, 38(3), 386–432. 10.1006/cogp.1998.0703
(