Skip to main content
Free AccessDiscussion Forum

Trust, but verify. The errors and misinterpretations in the Cochrane analysis by O. J. Storebo and colleagues on the efficacy and safety of methylphenidate for the treatment of children and adolescents with ADHD

Published Online:https://doi.org/10.1024/1422-4917/a000433

Abstract.Objective: A recent Cochrane review published by O. J. Storebo and colleagues (2015) raised substantial doubts about the benefit from stimulant medication with methylphenidate in the treatment of childhood ADHD due to the overall poor quality of studies. The systematic review thus contradicts all previous reviews and meta-analyses. Method: We here detail various examples of errors, inconsistencies, and misinterpretations in the review which led to false results and inadequate conclusions. Results:We demonstrate that the study selection is flawed and undertaken without sufficient scientific justification resulting in an underestimation of effect sizes, which, furthermore, are inadmissibly clinically interpreted. The methodology of the assessment of bias and quality is not objective and cannot be substantiated by the data. Conclusions: Cochrane reviews lay claim to a high scientific quality and substantial relevance for evidence-based clinical decisions. The systematic review by Storebo and colleagues (2015) illustrates that, despite adhering to strict standards and high-quality protocols, even Cochrane works should be critically read and verified, sometimes with surprising results.

Literature

  • Arnold, L. E., Lindsay, R. L., Conners, C. K., Wigal, S. B, Levine, A. J & Johnson, D. E., et al. (2004). A double-blind, placebo-controlled withdrawal trial of dexmethylphenidate hydrochloride in children with attention deficit hyperactivity disorder. Journal of Child and Adolescent Psychopharmacology, 14, 542–554. First citation in articleCrossref MedlineGoogle Scholar

  • Banaschewski, T. & Döpfner, M. (2014). DSM-5: Aufmerksamkeitsdefizit-/Hyperaktivitätsstörungen. Zeitschrift für Kinder- und Jugendpsychiatrie und Psychotherapie, 42, 271–277. First citation in articleLinkGoogle Scholar

  • Biederman, J., Quinn., D., Weiss, M., Markabi, S., Weidenman, M. & Edson, K., et al. (2003). Efficacy and safety of Ritalin LA, a new, once daily, extended-release dosage form of methylphenidate, in children with attention deficit hyperactivity disorder. Paediatric Drugs, 5, 833–841. First citation in articleCrossref MedlineGoogle Scholar

  • Bloch, M. H., Panza, K. E., Landeros-Weisenberger, A. & Leckman, J. F. (2009). Meta-analysis: treatment of attention-deficit/hyperactivity disorder in children with comorbid tic disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 48, 884–893. First citation in articleCrossref MedlineGoogle Scholar

  • Brown, R. T., Wynne, M. E. & Medenis, R. (1985). Methylphenidate and cognitive therapy: A comparison of treatment approaches with hyperactive boys. Journal of Abnormal Child Psychology, 13, 69–87. First citation in articleCrossref MedlineGoogle Scholar

  • Büchter, R. B. & Thomas, S. (2015). Re: Methylphenidate for attention-deficit/hyperactivity disorder in children and adolescents: Cochrane systematic review with meta-analyses and trial sequential analyses of randomised clinical trials; 10 December 2015; retrieved from http://www.bmj.com/content/351/bmj.h5203/rapid-responses. First citation in articleGoogle Scholar

  • Butter, H. J., Lapierre, Y., Firestone, P. & Blank, A. (1983). A comparative study of the efficacy of ACTH4-9 analog, methylphenidate, and placebo on attention deficit disorder with hyperkinesis. Journal of Clinical Psychopharmacology, 3, 226–230. First citation in articleCrossref MedlineGoogle Scholar

  • Carlson, G. A., Dunn, D., Kelsey, D., Ruff, D., Ball, S. & Ahrbecker, L., et al. (2007). A pilot study for augmenting atomoxetine with methylphenidate: Safety of concomitant therapy in children with attention-deficit/hyperactivity disorder. Child and Adolescent Psychiatry and Mental Health, 1, . First citation in articleCrossref MedlineGoogle Scholar

  • Charach, A., Yeung, E., Climans, T. & Lillie, E. (2011). Childhood attention-deficit/hyperactivity disorder and future substance use disorders: Comparative meta-analyses. Journal of the American Academy of Child and Adolescent Psychiatry, 50, 9–21. First citation in articleCrossref MedlineGoogle Scholar

  • Charach, A., Carson, P., Fox, S., Ali, M. U., Beckett, J. & Lim, C. G. (2013). Interventions for preschool children at high risk for ADHD: A comparative effectiveness review. Pediatrics, 131, 1584. First citation in articleCrossrefGoogle Scholar

  • Childress, A. C., Spencer, T., Lopez, F., Gerstner, O., Thulasiraman, A. & Muniz, R., et al. (2009). Efficacy and safety of dexmethylphenidate extended-release capsules administered once daily to children with attention-deficit/hyperactivity disorder. Journal of Child and Adolescent Psychopharmacology, 19, 351–361. First citation in articleCrossref MedlineGoogle Scholar

  • Coghill, D., Banaschewski, T., Lecendreux, M., Soutullo, C., Johnson, M. & Zuddas, A., et al. (2013). European, randomized, phase 3 study of lisdexamfetamine dimesylate in children and adolescents with attention-deficit/hyperactivity disorder. European Neuropsychopharmacology, 23, 1208–1218. First citation in articleCrossref MedlineGoogle Scholar

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.), Erlbaum, Hillsdale, NJ. First citation in articleGoogle Scholar

  • Döpfner, M., Banaschewski, T., Schmidt, J., Uebel, H., Schmeck, K. & Gerber, W. D., et al. (2003). Langzeitwirksames Methylphenidat bei Kindern mit Aufmerksamkeitsdefizit–Hyperaktivitätsstörungen: eine multizentrische Studie Long-acting methylphenidate preparation in children with ADHD – a multicenter study. Nervenheilkunde, 22, 85–92. First citation in articleGoogle Scholar

  • DuPaul, G. J. (1991). Parent and teacher ratings of ADHD symptoms: psychometric properties in a community-based sample. Journal of Clinical Child Psychology, 20, 245–253. First citation in articleCrossrefGoogle Scholar

  • Eichelberger, I., Plück, J., Hautmann, C., Hanisch, C. & Döpfner, M. (accepted). Effectiveness of the Prevention Program for Externalising Problem Behaviour (PEP) in preschoolers with severe ADHD symptoms and no or mild symptoms. Zeitschrift für Kinder- und Jugendpsychiatrie und Psychotherapie. First citation in articleMedlineGoogle Scholar

  • Faraone, S. V., Biederman, J. & Roe, C. (2002). Comparative efficacy of Adderall and methylphenidate in attention-deficit/hyperactivity disorder: a meta-analysis. Journal of Clinical Psychopharmacology, 22, 468–473. First citation in articleCrossref MedlineGoogle Scholar

  • Faraone, S. V., Biederman, J., Spencer, T. J. & Aleardi, M. (2006). Comparing the efficacy of medications for ADHD using meta-analysis. Medscape General Medicine, 8(4), . First citation in articleGoogle Scholar

  • Faraone, S. V. (2009). Using meta-analysis to compare the efficacy of medications for attention-deficit/hyperactivity disorder in youths. P & T: A Peer-Reviewed Journal for Formulary Management, 34, 678–694. First citation in articleMedlineGoogle Scholar

  • Faraone, S. V. & Glatt, S. J. (2010). A comparison of the efficacy of medications for adult attention-deficit/hyperactivity disorder using meta-analysis of effect sizes. Journal of Clinical Psychiatry, 71, 754–763. First citation in articleCrossref MedlineGoogle Scholar

  • Findling, R. L., Quinn, D., Hatch, S. J., Cameron, S. J., DeCory, H. H. & McDowell, M. (2006). Comparison of the clinical efficacy of twice-daily Ritalin and once-daily Equasym XL with placebo in children with attention deficit/hyperactivity disorder. European Child & Adolescent Psychiatry, 15, 450–459. First citation in articleCrossref MedlineGoogle Scholar

  • Findling, R. L., Bukstein, O. G., Melmed, R. D., López, F. A., Sallee, F. R. & Arnold, L. E., et al. (2008). A randomized, double-blind, placebo-controlled, parallel-group study of methylphenidate transdermal system in pediatric patients with ADHD. Journal of Clinical Psychiatry, 69, 149–159. Erratum in: Journal of Clinical Psychiatry, 69, 329. First citation in articleCrossref MedlineGoogle Scholar

  • Goodwin, G. (2015). Message from the President; December 2015. Retrieved from https://www.ecnp.eu/publications/presidents_blog/December%202015.aspx. First citation in articleGoogle Scholar

  • Grizenko, N., Qi Zhang, D. D., Polotskaia, A. & Joober, R. (2012). Efficacy of methylphenidate in ADHD children across the normal and the gifted intellectual spectrum. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 21, 282–288. First citation in articleMedlineGoogle Scholar

  • Hanwella, R., Senanayake, Mi. & De Silva, V. (2011). Comparative efficacy and acceptability of methylphenidate and atomoxetine in treatment of attention deficit hyperactivity disorder in children and adolescents: A meta-analysis. BMC Psychiatry, 11, . First citation in articleGoogle Scholar

  • Higgins, J. P. T. & Green, S. (Eds.) (2011). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, www.cochrane-handbook.org. First citation in articleGoogle Scholar

  • Ialongo, N. S., Lopez, M., Horn, W. F., Pascoe, J. M. & Greenberg, G. (1994). Effects of psychostimulant medication on self-perceptions of competence, control, and mood in children with attention deficit hyperactivity disorder. Journal of Clinical Child Psychology, 23, 161–173. First citation in articleCrossrefGoogle Scholar

  • Jacobi-Polishook, T., Shorer, Z. & Melzer, I. (2009). The effect of methylphenidate on postural stability under single and dual task conditions in children with attention deficit hyperactivity disorder – A double blind randomized control trial. Journal of the Neurological Sciences, 280(1–2), 15–21. First citation in articleCrossref MedlineGoogle Scholar

  • Kambeitz, J., Romanos, M. & Ettinger, U. (2014). Meta-analysis of the association between dopamine transporter genotype and response to methylphenidate treatment in ADHD. Pharmacogenomics Journal, 14(1), 77–84. First citation in articleCrossref MedlineGoogle Scholar

  • King, S., Griffin, S., Hodges, Z., Weatherly, H., Asseburg, C. & Richardson, G. et al. (2006). A systematic review and economic model of the effectiveness and cost-effectiveness of methylphenidate, dexamfetamine and atomoxetine for the treatment of attention deficit hyperactivity disorder in children and adolescents. Health Technology Assessment, 10(23), , . First citation in articleMedlineGoogle Scholar

  • Konrad, K., Günther, T., Hanisch, C. & Herpertz-Dahlmann, B. (2004). Differential effects of methylphenidate on attentional functions in children with attention-deficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 43, 191–198. First citation in articleCrossref MedlineGoogle Scholar

  • Konrad, K., Günther, T., Heinzel-Gutenbrunner, M. & Herpertz-Dahlmann, B. (2005). Clinical evaluation of subjective and objective changes in motor activity and attention in children with attention-deficit/hyperactivity disorder in a double-blind methylphenidate trial. Journal of Child and Adolescent Psychopharmacology, 15, 180–190. First citation in articleCrossref MedlineGoogle Scholar

  • Lundh, A., Sismondo, S., Lexchin, J., Busuioc, O. A. & Bero, L. (2012). Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews, 12, MR000033, DOI: 10.1002/ 14651858.MR000033.pub2. First citation in articleMedlineGoogle Scholar

  • Maia, C. R., Cortese, S., Caye, A., Deakin, T. K., Polanczyk, G. V. & Polanczyk, C. A., et al. (2010). Long-term efficacy of methylphenidate immediate-release for the treatment of childhood ADHD: A systematic review and meta-analysis. Journal of Attention Disorders [Epub ahead of print]. DOI: 10.1177/1087054714559643 [publication data should now be available]. First citation in articleGoogle Scholar

  • Mokros, L., Benien, N., Mütsch, A., Kinnen, C., Schürmann, S. & Wolf Metternich-Kaizman, T., et al. (2015). Angeleitete Selbsthilfe für Eltern von Kindern mit Aufmerksamkeitsdefizit-/Hyperaktivitätsstörung: Konzept, Inanspruchnahme und Effekte eines bundesweitern Angebotes – eine Beobachtungsstudie Guided self-help interventions for parents of children with ADHD-concept, referral and effectiveness in a nationwide trial. An observational study. Zeitschrift für Kinder- und Jugendpsychiatrie und Psychotherapie, 43, 275–288. DOI: 10.1024/1422-4917/a000348. First citation in articleLinkGoogle Scholar

  • Moshe, K., Karni, A. & Tirosh, E. (2012). Anxiety and methylphenidate in attention deficit hyperactivity disorder: A double-blind placebo-drug trial. Attention Deficit and Hyperactivity Disorders, 4, 153–158. First citation in articleCrossref MedlineGoogle Scholar

  • Newcorn, J. H., Kratochvil, C. J, Allen, A. J, Casat, C. D, Ruff, D. D & Moore, R. J, et al. (2008). Atomoxetine and osmotically released methylphenidate for the treatment of attention deficit hyperactivity disorder: Acute comparison and differential response. American Journal of Psychiatry, 165(6), 721–730. First citation in articleCrossref MedlineGoogle Scholar

  • Pliszka, S. R., Browne, R. G., Olvera, R. L. & Wynne, S. K. (2000). A double-blind, placebo-controlled study of Adderall and methylphenidate in the treatment of attention-deficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 619–626. First citation in articleCrossref MedlineGoogle Scholar

  • Punja, S., Zorzela, L., Hartling, L., Urichuk, L. & Vohra, S. (2013). Long-acting versus short-acting methylphenidate for paediatric ADHD: A systematic review and meta-analysis of comparative efficacy. BMJ Open, 3(3), . First citation in articleMedlineGoogle Scholar

  • Reichow, B., Volkmar, F. R. & Bloch, M. H. (2013). Systematic review and meta-analysis of pharmacological treatment of the symptoms of attention-deficit/hyperactivity disorder in children with pervasive developmental disorders. Journal of Autism and Developmental Disorders, 43, 2435–2441. First citation in articleCrossref MedlineGoogle Scholar

  • Riggs, P. D., Winhusen, T., Davies, R. D., Leimberger, J. D., Mikulich-Gilbertson, S. & Klein, C., et al. (2011). Randomized controlled trial of osmotic-release methylphenidate with cognitive-behavioral therapy in adolescents with attention deficit/hyperactivity disorder and substance use disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 50, 903–914. First citation in articleCrossref MedlineGoogle Scholar

  • Schachar, R. J., Tannock, R., Cunningham, C. & Corkum, P. V. (1997). Behavioral, situational, and temporal effects of treatment of ADHD with methylphenidate. Journal of the American Academy of Child and Adolescent Psychiatry, 36, 754–763. First citation in articleCrossref MedlineGoogle Scholar

  • Schachter, H. M., Phamm B., Kingm J. & Langfordm S., Moherm D. (2001). How efficacious and safe is short-acting methylphenidate for the treatment of attention-deficit disorder in children and adolescents? A meta-analysis. Canadian Medical Association Journal, 165, 1475–1488. First citation in articleMedlineGoogle Scholar

  • Seixas, M., Weiss, M. & Muller, U. (2012). Systematic review of national and international guidelines on attention-deficit hyperactivity disorder. Journal of Psychopharmacology, 26, 753–765. DOI: 10.1177/0269881111412095 First citation in articleCrossref MedlineGoogle Scholar

  • Stein, M. A., Waldman, I. D., Charney, E., Aryal, S., Sable, C. & Gruber, R., et al. (2011). Dose effects and comparative effectiveness of extended release dexmethylphenidate and mixed amphetamine salts. Journal of Child and Adolescent Psychopharmacology, 21, 581–588. First citation in articleCrossref MedlineGoogle Scholar

  • Storebo, O. J., Ramstad, E., Krogh, H. B., Nilausen, T. D., Skoog, M. & Holmskov, M., et al. (2015a). Methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD). Cochrane Database of Systematic Reviews, Issue 11. Art. No.: CD009885. DOI: 10.1002/14651858.CD009885.pub2. First citation in articleCrossref MedlineGoogle Scholar

  • Storebo, O. J., Krogh, H. B., Ramstad, E., Moreira-Maia, C. R., Holmskov, M. & Skoog, M., et al. (2015b). Methylphenidate for attention-deficit/hyperactivity disorder in children and adolescents: Cochrane systematic review with meta-analyses and trial sequential analyses of randomised clinical trials. British Medical Journal, 351, . DOI: 10.1136/bmj.h5203. First citation in articleMedlineGoogle Scholar

  • Storebo, O. J. (2015). Response to: Büchter, R. B., Thomas, S.: Re: Methylphenidate for attention-deficit/hyperactivity disorder in children and adolescents: Cochrane systematic review with meta-analyses and trial sequential analyses of randomised clinical trials, retrieved from http://www.bmj.com/content/351/bmj.h5203/rapid-responses. First citation in articleGoogle Scholar

  • Swanson, J. M., McBurnett, K., Wigal T., Pfiffner L. J., Lerner, M. A. & Williams, L., et al. (1993). Effect of stimulant medication on hyperactive children. A review of reviews. Exceptional Child, 60, 154–162. First citation in articleGoogle Scholar

  • Tourette’s Syndrome Study Group (2002). Treatment of ADHD in children with tics: A randomized controlled trial. Neurology, 58, 527–536. First citation in articleCrossref MedlineGoogle Scholar

  • Van der Oord, S., Prins, P. J. M., Oosterlaan, J. & Emmelkamp, P. M. G. (2008). Efficacy of methylphenidate, psychosocial treatments and their combination in school-aged children with ADHD: A meta-analysis. Clinical Psychology Review, 28, 783–800. First citation in articleCrossref MedlineGoogle Scholar

  • Van der Meere, J., Gunning, B. & Stemerdink, N. (1999). The effect of methylphenidate and clonidine on response inhibition and state regulation in children with ADHD. Journal of Child Psychology and Psychiatry and Allied Disciplines, 40, 291–298. First citation in articleCrossref MedlineGoogle Scholar

  • Wallace, A. E. & Kofoed, L. L. (1994). Statistical analysis of single case studies in the clinical setting: the example of methylphenidate trials in children with attention-deficit hyperactivity disorder. Journal of Child and Adolescent Psychopharmacology, 4, 141–150. First citation in articleCrossrefGoogle Scholar

  • Waxmonsky, J., Pelham, W. E., Gnagy, E., Cummings, M. R., O’Connor, B. & Majumdar, A., et al. (2008). The efficacy and tolerability of methylphenidate and behavior modification in children with attention-deficit/hyperactivity disorder and severe mood dysregulation. Journal of Child and Adolescent Psychopharmacology, 18, 573–588. First citation in articleCrossref MedlineGoogle Scholar

  • Wallander, J. L., Schroeder, S. R., Michelli, J. A. & Gualtieri, C. T. (1987). Classroom social interactions of attention deficit disorder with hyperactivity children as a function of stimulant medication. Journal of Pediatric Psychology, 12, 61–76. First citation in articleCrossref MedlineGoogle Scholar

  • Wolraich, M. L., Greenhill, L. L., Pelham, W., Swanson, J., Wilens, T. & Palumbo, D., et al. (2001). Randomized, controlled trial of OROS methylphenidate once a day in children with attention-deficit/hyperactivity disorder. Pediatrics, 108, 883–892. First citation in articleCrossref MedlineGoogle Scholar

  • Ullmann, R. K. & Sleator, E. K. (1986). Responders, nonresponders, and placebo responders among children with attention deficit disorder: importance of a blinded placebo evaluation. Clinical Pediatrics, 25, 594–599. First citation in articleCrossref MedlineGoogle Scholar

  • Zeni, C. P., Tramontina, S., Ketzer, C. R., Pheula, G. F. & Rohde, L. A. (2009). Methylphenidate combined with aripiprazole in children and adolescents with bipolar disorder and attention-deficit/hyperactivity disorder: A randomized crossover trial. Journal of Child and Adolescent Psychopharmacology, 19, 553–561. First citation in articleCrossref MedlineGoogle Scholar

  • Walitza, S., Renner, T. & Romanos, M. (2016). Aufmerksamkeits-Defizit/Hyperaktivitäts-Störung. In: Gerlach, M., Mehler-Wex, C., Walitza, S., Warnke, A., Wewetzer, C. (Eds.) Neuro-/Psychopharmaka im Kindes- und Jugendalter. Grundlagen und Therapie (3. Auflage). Springer, Wien Heidelberg, 415–428. First citation in articleCrossrefGoogle Scholar