Meta-Analysis of Binary Data Based upon Dichotomized Criteria
Abstract
Abstract. This paper considers meta-analysis of binary data that use a dichotomized continuous score. Classification into two categories, e.g., qualified or not qualified, is often based upon a threshold or cut-off value. This threshold might vary between studies since intentionally different values are used. However, conventional meta-analysis methodology analyzing sensitivity and specificity separately might then be confounded by a potentially unknown variation of the cut-off value. In order to cope with varying thresholds, an overall estimate of the misclassification error is suggested instead, which is equivalent to the well-known Youden index. It is argued that this index is less prone to between-study variation of cut-off values. To adjust for potential study effects a Mantel-Haenszel estimator of the overall misclassification error is suggested. Arguments are illustrated using, as an example, the diagnosis of alcoholism using the Alcohol Use Disorders Identification Test (AUDIT).
References marked with an asterisk (*) indicate studies included in the meta-analysis (see Table 2).
References
Babor, T.F. , de la Fuente, J.R. , Saunders, J. , Grant, M. (1989). International perspectives. From clinical research to secondary prevention: International collaboration in the development of the Alcohol Use Disorders Identification Test (AUDIT). Alcohol Health and Research World, 13, 371– 374 .Berner, M.M. , Kriston, L. , Bentele, M. , Härter, M. (2007). The Alcohol Use Disorders Identification Test for detecting at-risk drinking: A systematic review and meta-analysis. Journal of Studies on Alcohol and Drugs, 68, 1– 13 .*Bradley, K.A. , Bush, K.R. , McDonell, M.B. , Malone, T. , Fihn, S.D. (1998). For the Ambulatory Care Quality Improvement Project (ACQUIP). Screening for problem drinking: Comparison of CAGE and AUDIT. Journal of General Internal Medicine, 13, 379– 388 .*Contel, M. , Gual, A. , Colom, J. (1999). Alcohol Use Disorders Identification Test (AUDIT): Translation and validation to Catalan and Spanish (Spanish). Adicciones, 11, 337– 347 .Cooper, H. , Hedges, L. Eds. (1994). The handbook of research synthesis . New York: Russell Sage Foundation .Cruciani, M. , Marcati, P. , Malena, M. , Bosco, O. , Serpelloni, G. , Mengoli, C. (2002). Meta-analysis of diagnostic procedures for pneumocystis carninii pneumonia in HIV-1-infected patients. European Respiratory Journal, 20, 982– 989 .*Daeppen, J.-B. , Yersin, B. , Landry, U. , Pecoud, A. , Decrey, H. (2000). Reliability and validity of the Alcohol Use Disorders Identification Test (AUDIT) imbedded within a general health risk screening questionnaire: Results of a survey in 332 primary care patients. Alcoholism: Clinical and Experimental Research, 24, 659– 665 .DerSimonian, R. , Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177– 188 .Deville, W.L. , Buntinx, F. , Bouter, L.M. , Montori, V.M. , de Vet, H.C.W. , van der Windt, D.A.W.M. , et al. (2002). Conducting systematic reviews of diagnostic studies: Didactic guidelines. BMC Medical Research Methodology, 2, 9–Dukic, V. , Gatsonis, C. (2003). Meta-analysis of diagnostic test accuracy assessment studies with varying number of thresholds. Biometrics, 59, 936– 946 .Edwards, J.H. (1966). Some taxonomic implications of a curious feature of the bivariate normal surface. British Journal of Prevention and Social Medicine, 20, 42–*Gómez, A. , Conde, M. , Aguiar, J.A. , Santana, J.M. , Jorrin, A. , Betancor, P. (2001). Diagnostic usefulness of Alcohol Use Disorders Identification Test (AUDIT) for detecting hazardous alcohol consumption in primary care settings (Spanish). Medicina Clínica, 116, 121– 124 .*Gordon, A.J. , Maisto, S.A. , McNeil, M. , Kraemer, K.L. , Conigliaro, R.L. , Kelley, M.E. , et al. (2001). Three questions can detect hazardous drinkers. Journal of Family Practice, 50, 313– 320 .Greiner, M. (2003). Serodiagnostische Tests . [Serodiagnostic tests]. Berlin: Springer-Verlag .*Gual, A. , Segura, L. , Contel, M. , Heather, N. , Colom, J. (2002). AUDIT-3 and AUDIT-4: Effectiveness of two short forms of the Alcohol Use Disorders Identification Test. Alcohol and Alcoholism, 37, 591– 596 .Haddock, K.C. , Rindskopf, D. , Shadish, W.R. Jr. (1998). Using odds ratios as effect sizes for meta-analysis of dichotomous data: A primer on methods and issues. Psychological Methods, 3, 339– 353 .Harbord, R.M. , Deeks, J.J. , Egger, M. , Whiting, P. , Sterne, J.A.C. (2007). A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics, 8, 239– 251 .Hasselblad, V. , Hedges, L.V. (1995). Meta-analysis of screening and diagnostic tests. Psychological Bulletin, 117, 167– 178 .Higgins, J.P. , Thompson, S.G. , Deeks, J.J. , Altmann, D.G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327, 557– 560 .*Hiro, H. , Shima, S. (1996). Availability of the Alcohol Use Disorders Identification Test (AUDIT) for a complete health examination in Japan. Japanese Journal of Alcoholism. Studies and Drug Dependence, 31, 437– 450 .Hunter, J.E. , Schmidt, F.L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage .Irwig, L. , Tosteson, A.N. , Gatsonis, C. , Lau, J. , Colditz, G. , Chalmers, T.C. , et al. (1994). Guidelines for meta-analyses evaluating diagnostic tests. Annals of Internal Medicine, 120, 667– 676 .Irwig, L. , Macaskill, P. , Glasziou, P. , Fahey, M. (1995). Meta-analytic methods for diagnostic test accuracy. Journal of Clinical Epidemiology, 48, 119– 130 .*Kokotailo, P.K. , Egan, J. , Gangnon, R. , Brown, D. , Mundt, M. , Fleming, M. (2004). Validity of the Alcohol Use Disorders Identification Test in college students. Alcoholism: Clinical and Experimental Research, 28, 914– 920 .Lijmer, J.G. , Bossuyt, P.M.M. , Heisterkamp, S.H. (2002). Exploring sources of heterogeneity in systematic reviews of diagnostic tests. Statistics in Medicine, 21, 1525– 1537 .*MacKenzie, D. , Langa, A. , Brown, T.M. (1996). Identifying hazardous or harmful alcohol use in medical admissions: A comparison of AUDIT, CAGE and brief MAST. Alcohol and Alcoholism, 31, 591– 599 .McCullagh, P. (1980). Regression models for ordinal data. Journal of the Royal Statistical Society, Series B, 42, 109– 142 .Midgette, A.S. , Stukel, T.A. , Littenberg, B. (1993). A meta-analytic method for summarizing diagnostic test performances: Receiver-operating-characteristic-summary point estimates. Medical Decision Making, 13, 253– 257 .*Neumann, T. , Neuner, B. , Gentilello, L.M. , Weiss-Gerlach, E. , Mentz, H. , Rettig, J.S. (2004). Gender differences in the performance of a computerized version of the Alcohol Use Disorders Identification Test in subcritically injured patients who are admitted to the emergency department. Alcoholism: Clinical and Experimental Research, 28, 1693– 1701 .Pepe, M.S. (2000). Receiver operating characteristic methodology. Journal of the American Statistical Association, 95, 308– 311 .*Philpot, M. , Pearson, N. , Petratou, V. , Dayanandan, R. , Silverman, M. , Marshall, J. (2003). Screening for problem drinking in older people referred to a mental health service: A comparison of CAGE and AUDIT. Aging and Mental Health, 7, 171– 175 .*Piccinelli, M. , Tessari, E. , Bortolomasi, M. , Piasere, O. , Semenzi, M. , Garzotto, N. , et al. (1997). Efficacy of the Alcohol Use Disorders Identification Test as a screening tool for hazardous alcohol intake and related disorders in primary care: A validity study. British Medical Journal, 314, 420– 424 .Reitsma, J.B. , Glas, A.S. , Rutjes, A.W.S. , Scholten, R.J.P.M. , Bossuyt, P.M. , Zwinderman, A.H. (2005). Bivariate analysis of sensitivity and specificity produces informative measures in diagnostic reviews. Journal of Clinical Epidemiology, 58, 982– 990 .*Rumpf, H.J. , Hapke, U. , Meyer, C. , John, U. (2002). Screening for alcohol use disorders and at-risk drinking in the general population: Psychometric performance of three questionnaires. Alcohol and Alcoholism, 37, 261– 268 .Rutter, C.M. , Gatsonis, C.A. (2001). A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Statistics in Medicine, 20, 2865– 2884 .Sánchez-Meca, J. , Marín-Martínez, F. , Chacón-Moscoso, S. (2003). Effect-size indices for dichotomized outcomes in meta-analysis. Psychological Methods, 8, 448– 467 .Schulze, R. , Holling, H. , Böhning, D. Eds. (2003). Meta-analysis. New developments and applications in medical and social sciences . Seattle, WA: Hogrefe & Huber .Shields, A.L. , Caruso, J.C. (2003). Reliability generalization of the Alcohol Use Disorders Identification Test. Educational and Psychological Measurement, 63, 404– 413 .*Skipsey, K. , Burleson, J.A. , Kranzler, H.R. (1997). Utility of the AUDIT for identification of hazardous or harmful drinking in drug-dependent patients. Drug and Alcohol Dependence, 45, 157– 163 .Sutton, A.J. , Abrams, K.R. , Jones, D.R. , Sheldon, T.A. , Song, F. (2000). Methods for meta-analysis in medical research . New York: Wiley .*Taj, N. , Devera-Sales, A. , Vinson, D.C. (1998). Screening for problem drinking: Does a single question work?. Journal of Family Practice, 46, 328– 335 .Tosteson, A. , Begg, C. (1988). A general regression methodology for ROC curve estimation. Medical Decision Making, 8, 204– 215 .Van Houwelingen, H.C. , Zwinderman, K.H. , Stijnen, T. (1993). A bivariate approach to meta-analysis. Statistics in Medicine, 12, 2273– 2284 .Youden, D. (1950). Index for rating diagnostic tests. Cancer, 3, 32– 35 .Zou, K. , Hall, W. , Shapiro, D. (1997). Smooth nonparametric ROC curves for continuous diagnostic tests. Statistics in Medicine, 16, 2143– 2156 .
