Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigation
Previous article
Next article
Original article

Retest reliabilities of decision-making and cognitive control measures in addictive disorders

Published Online:August 30, 2016https://doi.org/10.1024/0939-5911/a000430

Abstract

Abstract.Aims: Longitudinal and intervention studies are needed on impaired decision-making and cognitive control deficits as putative risk factors for Substance-Related and Addictive Disorders (SAD). Adequate stability of measures is essential for this approach. To improve our knowledge, we aimed 1) to analyse retest reliabilities of such behavioural measures and 2) to compare retest reliabilities between SAD and controls. Methods: In a quasi-experimental design we recruited a convenience sample of three groups: A Gambling Disorder group (n = 26), a Nicotine Dependence group (n = 42), both diagnosed according to DSM-IV, and a healthy control group (n = 52). Participants performed two test sessions within 3 – 4 weeks with six tasks assessing decision-making and cognitive control. Results: Retest reliabilities, indicated by intraclass correlation coefficients, varied extremely between tasks and parameters ranging from 0.31 (poor) to 0.82 (excellent) with the majority ranging from 0.40 (fair) to 0.74 (good). Importantly, retest reliabilities differed significantly between the SAD groups and the control group. Conclusions: Retest reliabilities of decision-making and cognitive control measures are adequate for longitudinal and intervention studies of SAD, although tasks parameters should be selected carefully. However, group differences in retest reliabilities may result in misleading group and intervention effects. To minimize measurement error, studies investigating longitudinal designs may apply latent variable models.


Retest-Reliabilität von Verhaltensmaßen bei Abhängigkeitsstörungen

Zusammenfassung.Ziele: Defizite im Entscheidungsverhalten und in kognitiver Kontrolle sind mögliche Risikofaktoren für Störungen durch Substanzkonsum und abhängige Verhaltensweisen (SAD). Längsschnitt- und Interventionsstudien zu diesen Fragestellungen setzen eine adäquate Stabilität der entsprechenden Messinstrumente voraus. Wir haben dazu 1) die Retest-Reliabilität dazugehöriger Verhaltensmaße analysiert und 2) die Retest-Reliabilität zwischen SAD und Kontrollen verglichen. Methode: In einem quasi-experimentellen Design rekrutierten wir eine Gelegenheitsstichprobe folgender drei Gruppen: Eine Gruppe mit einer Störung durch Glücksspielen (n = 26) und eine mit Nikotinabhängigkeit (n = 42), jeweils diagnostiziert nach DSM-IV, sowie ein gesunde Kontrollgruppe (n = 52). Die Teilnehmer bearbeiteten an zwei Untersuchungszeitpunkten innerhalb von 3 – 4 Wochen jeweils 6 Aufgaben zur Erfassung von Entscheidungsverhalten und kognitiver Kontrolle. Ergebnisse: Die Retest-Reliabilität, erfasst mit dem Intraklassen-Korrelationskoeffizienten, variierte stark zwischen den Aufgaben und deren Parametern von 0.31 (schlecht) bis 0.82 (exzellent), wobei die meisten zwischen 0.40 (akzeptabel) und 0.74 (gut) lagen. Weiterhin unterschied sich die Retest-Reliabilität signifikant zwischen den SAD-Gruppen und der Kontrollgruppe. Schlussfolgerungen: Die Retest-Reliabilität von Verhaltensmaßen zur Erfassung von Entscheidungsverhalten und kognitiver Kontrolle ist adäquat für Längsschnitt- und Interventionsstudien bei SAD, wobei die Aufgabenparameter sorgfältig ausgewählt werden sollten. Es ist wichtig in solchen Studiendesigns zu beachten, dass Gruppenunterschiede in der Retest-Reliabilität der Instrumente zu falschen Gruppen- und Interventionseffekten führen können. Um den Messfehler zu verringern, werden bei Längsschnittdesigns latente Variablenmodelle empfohlen.

References

  • Aichert, D. S., Wöstmann, N. M., Costa, A., Macare, C., Wenig, J. R., & Möller, H.-J.et al. (2012). Associations between trait impulsivity and prepotent response inhibition. Journal of Clinical and Experimental Neuropsychology, 34 (10), 1016 – 1032. First citation in articleCrossref, Google Scholar

  • American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders: DSM-IV-TR (4th, text revision). Washington, DC: American Psychiatric Association. First citation in articleGoogle Scholar

  • Beck, S. M., Ruge, H., Schindler, C., Burkart, M., Miller, R., & Kirschbaum, C.et al. (2015). Effects of Ginkgo biloba extract EGb 761® on cognitive control functions, mental activity of the prefrontal cortex and stress reactivity in elderly adults with subjective memory impairment – a randomized double-blind placebocontrolled trial. Human Psychopharmacology: Clinical and Experimental, 31 (3), 227 – 242. First citation in articleGoogle Scholar

  • Black, A. C., & Rosen, M. I. (2011). A money management-based substance use treatment increases valuation of future rewards. Addictive Behaviors, 36 (1 – 2), 125 – 128. First citation in articleCrossref, Google Scholar

  • Borgaro, S., Pogge, D. L., DeLuca, V. A., Bilginer, L., Stokes, J., & Harvey, P. D. (2003). Convergence of different versions of the Continuous Performance Test: Clinical and scientific implications. Journal of Clinical & Experimental Neuropsychology, 25 (2), 283 – 292. First citation in articleCrossref, Google Scholar

  • Braver, T. S., Satpute, A. B., Rush, B. K., Racine, C. A., & Barch, D. M. (2005). Context processing and context maintenance in healthy aging and early stage dementia of the Alzheimer’s type. Psychology and Aging, 20 (1), 33 – 46. First citation in articleCrossref, Google Scholar

  • Bühringer, G., Wittchen, H.-U., Gottlebe, K., Kufeld, C., & Goschke, T. (2008). Why people change? The role of cognitive control processes in the onset and cessation of substance abuse disorders. International Journal of Methods in Psychiatric Research, 17 (S1), 4 – 15. First citation in articleGoogle Scholar

  • Cicchetti, D. V., & Sparrow, S. A. (1981). Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency, 86 (2) ,127 – 137. First citation in articleGoogle Scholar

  • Clark, L., Cools, R., & Robbins, T. W. (2004). The neuropsychology of ventral prefrontal cortex: decision-making and reversal learning. Brain and Cognition, 55, 41 – 53. First citation in articleCrossref, Google Scholar

  • Clark, L., Manes, F., Antoun, N., Sahakian, B. J., & Robbins, T. W. (2003). The contributions of lesion laterality and lesion volume to decision-making impairment following frontal lobe damage. Neuropsychologia, 41, 1474 – 1483. First citation in articleCrossref, Google Scholar

  • Congdon, E., Mumford, J. A., Cohen, J. R., Galvan, A., Canli, T., & Poldrack, R. A. (2012). Measurement and reliability of response inhibition. Frontiers in Psychology, 3 (37), 1 – 10. First citation in articleCrossref, Google Scholar

  • Cools, R., Sheridan, M., Jacobs, E., & D’Esposito, M. (2007). Impulsive personality predicts dopamine-dependent changes in rrontostriatal activity during component processes of working memory. The Journal of Neuroscience, 27 (20), 5506 – 5514. First citation in articleCrossref, Google Scholar

  • Fellows, L. K., & Farah, M. J. (2005). Different underlying impairments in decision-making following ventromedial and dorsolateral frontal lobe damage in humans. Cerebral Cortex, 15 (1), 58 – 63. First citation in articleCrossref, Google Scholar

  • Fernie, G., Peeters, M., Gullo, M. J., Christiansen, P., Cole, J. C., & Sumnall, H.et al. (2013). Multiple behavioural impulsivity tasks predict prospective alcohol involvement in adolescents. Addiction, 108 (11), 1916 – 1923. First citation in articleCrossref, Google Scholar

  • Friedman, N. P., Miyake, A., Young, S. E., DeFries, J. C., Corley, R. P., & Hewitt, J. K. (2008). Individual differences in executive functions are almost entirely genetic in origin. Journal of Experimental Psychology: General, 137 (2), 201 – 225. First citation in articleCrossref, Google Scholar

  • Goldstein, R. Z., & Volkow, N. D. (2011). Dysfunction of the prefrontal cortex in addiction: neuroimaging findings and clinical implications. Nature Reviews Neuroscience, 12 (11), 652 – 669. First citation in articleCrossref, Google Scholar

  • Goschke, T. (2014). Dysfunctions of decision-making and cognitive control as transdiagnostic mechanisms of mental disorders: advances, gaps, and needs in current research. International Journal of Methods in Psychiatric Research, 23 (S1), 41 – 57. First citation in articleCrossref, Google Scholar

  • Houben, K., Havermans, R. C., Nederkoorn, C., & Jansen, A. (2012). Beer à no-go: learning to stop responding to alcohol cues reduces alcohol intake via reduced affective associations rather than increased response inhibition. Addiction, 107 (7), 1280 – 1287. First citation in articleCrossref, Google Scholar

  • Kindlon, D., Mezzacappa, E., & Earls, F. (1995). Psychometric properties of impulsivity measures: Temporal stability, validity and factor structure. Journal of Child Psychology and Psychiatry, 36 (4), 645 – 661. First citation in articleCrossref, Google Scholar

  • Kindt, M., Bierman, D., & Brosschot, J. F. (1996). Stroop versus Stroop: Comparison of a card format and a single-trial format of the standard color-word Stroop task and the emotional Stroop task. Personality and Individual Differences, 21 (5), 653 – 661. First citation in articleCrossref, Google Scholar

  • Kirby, K. N., Petry, N. M., & Bickel, W. K. (1999). Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls. Journal of Experimental Psychology: General, 128 (1), 78 – 87. First citation in articleCrossref, Google Scholar

  • Kräplin, A., Behrendt, S., Scherbaum, S., Dshemuchadse, M., Bühringer, G., & Goschke, T. (2015). Increased impulsivity in pathological gambling: Considering nicotine dependence. Journal of Clinical and Experimental Neuropsychology, 37 (4), 367 – 378. First citation in articleCrossref, Google Scholar

  • Kräplin, A., Dshemuchadse, M., Behrendt, S., Scherbaum, S., Goschke, T., & Bühringer, G. (2014). Dysfunctional decision-making in pathological gambling: Pattern specificity and the role of impulsivity. Psychiatry Research, 215 (3), 675 – 682. First citation in articleCrossref, Google Scholar

  • Kray, J., Karbach, J., Haenig, S., & Freitag, C. (2011). Can task-switching training enhance executive control functioning in children with attention deficit/-hyperactivity disorder? Frontiers in Human Neuroscience, 5, 1 – 9. First citation in articleGoogle Scholar

  • Kuntsi, J., Stevenson, J., Oosterlaan, J., & Sonuga-Barke, E. J. S. (2001). Test-retest reliability of a new delay aversion task and executive function measures. British Journal of Developmental Psychology, 19 (3), 339 – 348. First citation in articleCrossref, Google Scholar

  • Lane, S. D., Cherek, D. R., Lieving, L. M., & Tcheremissine, O. V. (2005). Marijuana effects on human forgetting functions. J Exp Anal Behav, 83 (1), 67 – 83. First citation in articleCrossref, Google Scholar

  • Lehrl, S. (2005). Mehrfachwahl-Wortschatz-Intelligenztest MWT-B (5.). Balingen: Spitta Verlag. First citation in articleGoogle Scholar

  • Lemay, S., Bédard, M.-A., Rouleau, I., & Tremblay, P.-L. (2004). Practice effect and test-retest reliability of attentional and executive tests in middle-aged to elderly subjects. The Clinical Neuropsychologist, 18 (2), 284 – 302. First citation in articleCrossref, Google Scholar

  • Logan, G. D., Schachar, R. J., & Tannock, R. (1997). Impulsivity and inhibitory control. Psychological Science, 8, 60 – 64. First citation in articleCrossref, Google Scholar

  • Lowe, C., & Rabbitt, P. (1998). Test\re-test reliability of the CANTAB and ISPOCD neuropsychological batteries: theoretical and practical issues. Neuropsychologia, 36 (9), 915 – 923. First citation in articleCrossref, Google Scholar

  • MacKillop, J., Amlung, M. T., Few, L. R., Ray, L. A., Sweet, L. H., & Munafo, M. R. (2011). Delayed reward discounting and addictive behavior: a meta-analysis. Psychopharmacology (Berlin), 216 (3), 305 – 321. First citation in articleCrossref, Google Scholar

  • Miyake, A., & Friedman, N. P. (2012). The nature and organization of individual differences in executive functions: Four general conclusions. Current Directions in Psychological Science, 21 (1), 8 – 14. First citation in articleCrossref, Google Scholar

  • Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. P. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: a latent variable analysis. Cognitive Psychology, 41, 49 – 100. First citation in articleCrossref, Google Scholar

  • Nigg, J. T., Wong, M. M., Martel, M. M., Jester, J. M., Puttler, L. I., & Glass, J. M.et al. (2006). Poor response inhibition as a predictor of problem drinking and illicit drug use in adolescents at risk for alcoholism and other substance use disorders. Journal of the American Academy of Child & Adolescent Psychiatry, 45 (4), 468 – 475. First citation in articleCrossref, Google Scholar

  • Overall, J., & Woodward, J. (1975). Unreliability of differences scores: A paradox for measurement of change. Psychological Bulletin, 82, 85 – 86. First citation in articleCrossref, Google Scholar

  • Peeters, M., Janssen, T., Monshouwer, K., Boendermaker, W., Pronk, T., & Wiers, R.et al. (2015). Weaknesses in executive functioning predict the initiating of adolescents’ alcohol use. Developmental Cognitive Neuroscience, 16, 139 – 146. First citation in articleCrossref, Google Scholar

  • Pomerleau, C. S., Pomerleau, O. F., Snedecor, S. M., & Mehringer, A. M. (2004). Defining a never-smoker: Results from the nonsmokers survey. Addictive Behaviors, 29 (6), 1149 – 1154. First citation in articleCrossref, Google Scholar

  • Rogers, R. D., Everitt, B. J., Baldacchino, A., Blackshaw, A. J., Swainson, R., & Wynne, K.et al. (1999). Dissociable deficits in the decision-making cognition of chronic amphetamine abusers, opiate abusers, patients with focal damage to prefrontal cortex, and tryptophan-depleted normal volunteers: evidence for monoaminergic mechanisms. Neuropsychopharmacology, 20 (4), 322 – 339. First citation in articleCrossref, Google Scholar

  • Rubia, K., Russell, T., Overmeyer, S., Brammer, M. J., Bullmore, E. T., & Sharma, T.et al. (2001). Mapping Motor Inhibition: Conjunctive Brain Activations across Different Versions of Go/No-Go and Stop Tasks. NeuroImage, 13 (2), 250 – 261. First citation in articleCrossref, Google Scholar

  • Rubia, K., Smith, A. B., Brammer, M. J., Toone, B., & Taylor, E. (2005). Abnormal brain activation during inhibition and error detection in medication-naive adolescents with ADHD. American Journal of Psychiatry, 162, 1067 – 1075. First citation in articleCrossref, Google Scholar

  • Simpson, C. A., & Vuchinich, R. E. (2000). Reliability of a measure of temporal discounting. The Psychological Record, 50, 3 – 16. First citation in articleCrossref, Google Scholar

  • Slutske, W. S., Caspi, A., Moffitt, T. E., & Poulton, R. (2005). Personality and problem gambling – A prospective study of a birth cohort of young adults. Archives of General Psychiatry, 62, 769 – 775. First citation in articleCrossref, Google Scholar

  • Smith, J. L., Mattick, R. P., Jamadar, S. D., & Iredale, J. M. (2014). Deficits in behavioural inhibition in substance abuse and addiction: A meta-analysis. Drug and Alcohol Dependence, 145 (0), 1 – 33. First citation in articleCrossref, Google Scholar

  • Soreni, N., Crosbie, J., Ickowicz, A., & Schachar, R. (2009). Stop Signal and Conners’ Continuous Performance Tasks: Test—retest reliability of two inhibition measures in ADHD children. Journal of Attention Disorders, 13 (2), 137 – 143. First citation in articleCrossref, Google Scholar

  • Stata Corp. (2014). Stata Statistical Software: Release 14.0. Stata Corporation. First citation in articleGoogle Scholar

  • Strauss, G. P., Allen, D. N., Jorgensen, M. L., & Cramer, S. L. (2005). Test-retest reliability of standard and emotional Stroop tasks: An investigation of color-word and picture-word versions. Assessment, 12 (3), 330 – 337. First citation in articleCrossref, Google Scholar

  • van Holst, R. J., van den Brink, W., Veltman, D. J., & Goudriaan, A. E. (2010). Why gamblers fail to win: a review of cognitive and neuroimaging findings in pathological gambling. Neuroscience & Biobehavioral Reviews, 34, 87 – 107. First citation in articleCrossref, Google Scholar

  • Vaz, S., Falkmer, T., Passmore, A. E., Parsons, R., & Andreou, P. (2013). The case for using the repeatability coefficient when calculating test–retest reliability. PLoS ONE, 8 (9),e73990. First citation in articleGoogle Scholar

  • Verdejo-Garcia, A., Lawrence, A. J., & Clark, L. (2008). Impulsivity as a vulnerability marker for substance-use disorders: review of findings from high-risk research, problem gamblers and genetic association studies. Neuroscience & Biobehavioral Reviews, 32, 777 – 810. First citation in articleCrossref, Google Scholar

  • Weafer, J., Baggott, M. J., & de Wit, H. (2013). Test–retest reliability of behavioral measures of impulsive choice, impulsive action, and inattention. Experimental and Clinical Psychopharmacology, 21 (6), 475 – 481. First citation in articleCrossref, Google Scholar

  • Wiehler, A., & Peters, J. (2015). Reward-based decision making in pathological gambling: The roles of risk and delay. Neuroscience Research, 90, 3 – 14. First citation in articleCrossref, Google Scholar

  • Wiers, R. W., Gladwin, T. E., Hofmann, W., Salemink, E., & Ridderinkhof, K. R. (2013). Cognitive bias modification and cognitive control training in addiction and related psychopathology: Mechanisms, clinical perspectives, and ways forward. Clinical Psychological Science, 1 (2), 192 – 212. First citation in articleCrossref, Google Scholar

  • Wilcox, R. R., & Keselman, H. J. (2003). Modern robust data analysis methods: Measures of central tendency. Psychological Methods, 8 (3), 254 – 274. First citation in articleCrossref, Google Scholar

  • Wittchen, H.-U., & Pfister, H. (1997). DIA-X Interview. Instruktionsmanual zur Durchführung von DIA-X-Interviews [Instruction manual for the DIA-X-Interview]. Frankfurt: Swets & Zeitlinger. First citation in articleGoogle Scholar

  • Wöstmann, N. M., Aichert, D. S., Costa, A., Rubia, K., Möller, H.-J., & Ettinger, U. (2013). Reliability and plasticity of response inhibition and interference control. Brain and Cognition, 81 (1), 82 – 94. First citation in articleCrossref, Google Scholar