Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Setting Doesn’t Matter Much

A Meta-Analytic Comparison of the Results of Intelligence Tests Obtained in Group and Individual Settings

Published Online:April 07, 2017https://doi.org/10.1027/1015-5759/a000402

Abstract

Abstract. This study deals with the effects of the diagnostic setting on the performance in intelligence tests. We conducted a meta-analysis in which k = 30 samples with a total sample size of N = 2,448 were integrated. We compared results for the same intelligence tests administered either in a group or in an individual setting. The main analysis indicated a small mean population effect [M(g) = 0.085] that was not significant [−0.036 ≤ M(g) ≤ 0.206]. Nevertheless, moderator analyses indicated a stronger [M(g) = 0.193] and significant [0.087 ≤ M(g) ≤ 0.298] effect in favor of individual settings for studies employing a between-person design. Setting effects in within-person designs were most likely superimposed by retest effects. As the setting effect was very small, the current testing practice in which results obtained in group and individual settings are treated as interchangeable is not overly problematic. However, our results encourage test developers to examine setting effects before stating that results obtained in different settings are equivalent. Between-person designs using participants of comparable ability are most suitable in this context as retest effects can be ruled out.

References Studies included in the meta-analysis are marked with an asterisk (*).

  • Allport, F. H. (1920). The influence of the group upon association and thought. Journal of Experimental Psychology, 3, 159–182. doi: 10.1037/h0067891 First citation in articleCrossref, Google Scholar

  • Anderson, C. A. (1929). An experimental study of social facilitation as affected by intelligence. American Journal of Sociology, 34, 874–881. doi: 10.1086/214828 First citation in articleCrossref, Google Scholar

  • Arthur, W., Bennett, W., & Huffcutt, A. I. (2001). Conducting meta-analysis using SAS. Mahwah, NJ: Erlbaum. First citation in articleCrossref, Google Scholar

  • *Bennett, M. W. (1946). Factors influencing performance on group and individual tests of intelligence: II. Social facilitation. Journal of Educational Psychology, 37, 347–358. doi: 10.1037/h0059535 First citation in articleCrossref, Google Scholar

  • *Bobele, R. M. (1976). Efficacy of the visual retention test as a group-administered instrument for young children. Perceptual and Motor Skills, 43, 267–272. doi: 10.2466/pms.1976.43.1.267 First citation in articleCrossref, Google Scholar

  • Bond, C. F., & Titus, L. J. (1983). Social facilitation: A meta-analysis of 241 studies. Psychological Bulletin, 94, 265. doi: 10.1037/0033-2909.94.2.265 First citation in articleCrossref, Google Scholar

  • Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester, UK: Wiley. First citation in articleCrossref, Google Scholar

  • *Brannigan, G. G., & Brannigan, M. J. (1995). Comparison of individual versus group administration of the modified version of the Bender-Gestalt Test. Perceptual and Motor Skills, 80, 1274. doi: 10.2466/pms.1995.80.3c.1274 First citation in articleCrossref, Google Scholar

  • *Caskey, W. E., & Larson, G. L. (1977). Two modes of administration of the Bender Visual-Motor Gestalt Test to kindergarten children. Perceptual and Motor Skills, 45, 1003–1006. doi: 10.2466/pms.1977.45.3.1003 First citation in articleCrossref, Google Scholar

  • Cox, W. M., & Blount, J. P. (1999). Individual versus group administration of the memory-for-designs test to alcoholics. Alcoholism Treatment Quarterly, 16, 113–118. doi: 10.1300/J020v16n04_10 First citation in articleCrossref, Google Scholar

  • *Curtis, C. J., Michael, J. J., & Michael, W. B. (1979). The predictive validity of the Developmental Test of Visual-Motor Integration under group and individual modes of administration relative to academic performance measures of second-grade pupils without identifiable major learning disabilities. Educational and Psychological Measurement, 39, 401–410. doi: 10.1177/001316447903900219 First citation in articleCrossref, Google Scholar

  • Dandurand, F., Shultz, T. R., & Onishi, K. H. (2008). Comparing online and lab methods in a problem-solving experiment. Behavior Research Methods, 40, 428–434. doi: 10.3758/BRM.40.2.428 First citation in articleCrossref, Google Scholar

  • Dashiell, J. F. (1930). An experimental analysis of some group effects. The Journal of Abnormal and Social Psychology, 25, 190–199. doi: 10.1037/h0075144 First citation in articleCrossref, Google Scholar

  • Eisenthal, S., & Harford, T. (1969). Variation in the form and administration of Raven’s Progressive Matrices Scale in a neuropsychiatric population. Psychological Reports, 24, 262. doi: 10.2466/pr0.1969.24.1.262 First citation in articleCrossref, Google Scholar

  • *Eme, R. F. (1968). The WAIS as a group test of intelligence. Chicago, IL: Loyola University Chicago. First citation in articleGoogle Scholar

  • *Fargo, G. A., Crowell, D. C., Noyes, M. H., Fuchigami, R. Y., Gordon, J. M., & Dunn-Rankin, P. (1967). Comparability of group television and individual administration of the Peabody Picture Vocabulary Test: Implications for screening. Journal of Educational Psychology, 58, 137–140. doi: 10.1037/h0024653 First citation in articleCrossref, Google Scholar

  • *Farnsworth, P. R. (1928). Concerning so-called group effects. The Pedagogical Seminary and Journal of Genetic Psychology, 35, 587–594. doi: 10.1080/08856559.1928.10532174 First citation in articleCrossref, Google Scholar

  • Freeman, F. S. (1923). A comparison of IQ’s Obtained with Dearborn Group Tests and the Stanford Revision. Journal of Educational Psychology, 14, 441–443. doi: 10.1037/h0076034 First citation in articleCrossref, Google Scholar

  • *Goldmann, J. (2011). Einzel- versus Gruppenvorgabe der Untertests Antonyme, Synonyme finden und Funktionen Abstrahieren des AID 3 [Individual- versus group administration of the subtests finding antonyms, synonyms and abstracting functions of the AID-3]. Wien, Austria: Universität Wien. First citation in articleGoogle Scholar

  • Goldstein, D. J. (1978). Group and individual administration of the Wickens technique. Psychological Reports, 42, 111–114. doi: 10.2466/pr0.1978.42.1.111 First citation in articleCrossref, Google Scholar

  • Guérin, B. (1993). Social facilitation. Cambridge, UK: Cambridge University Press. First citation in articleCrossref, Google Scholar

  • Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486–504. doi: 10.1037/1082-989X.3.4.486 First citation in articleCrossref, Google Scholar

  • Henmi, M., & Copas, J. B. (2010). Confidence intervals for random effects meta-analysis and robustness to publication bias. Statistics in Medicine, 29, 2969–2983. doi: 10.1002/sim.4029 First citation in articleCrossref, Google Scholar

  • Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ, 327, 557–560. doi: 10.1136/bmj.327.7414.557 First citation in articleCrossref, Google Scholar

  • Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks, CA: Sage. First citation in articleCrossref, Google Scholar

  • Husband, R. W. (1931). Analysis of methods in human maze learning. The Pedagogical Seminary and Journal of Genetic Psychology, 39, 258–278. doi: 10.1080/08856559.1931.10532308 First citation in articleCrossref, Google Scholar

  • *Ihme, J. M., Lemke, F., Lieder, K., Martin, F., Müller, J. C., & Schmidt, S. (2009). Comparison of ability tests administered online and in the laboratory. Behavior Research Methods, 41, 1183–1189. doi: 10.3758/BRM.41.4.1183 First citation in articleCrossref, Google Scholar

  • *Ireland, J. F. (1974). A comparison of individual and group administration of the Cattell Culture Fair Intelligence Scale and the Wechsler Intelligence Scale for Children in a delinquent population. Tucson, AZ: University of Arizona. First citation in articleGoogle Scholar

  • Jacobs, J. C. (1971). Group administration of the Bender Gestalt. Psychology in the Schools, 8, 345–346. doi: 10.1002/1520-6807(197110)8:4<345::AID-PITS2310080410>3.0.CO;2-Z First citation in articleCrossref, Google Scholar

  • Keogh, B. K., & Vormeland, O. (1970). Performance of Norwegian children on the Bender Gestalt and Draw‐a‐Person Tests. Pedagogisk Forskning, 14, 105–111. doi: 10.1080/0031383700140107 First citation in articleCrossref, Google Scholar

  • *Kornmann, R., Endrigkeit, F., & Sander, H. (1972). Sind lernbehinderte Sonderschüler in Gruppen-Intelligenztests benachteiligt? [Are children with learning disabilities disadvantaged by group intelligence tests?] Diagnostica, 3, 111–121. First citation in articleGoogle Scholar

  • *Krueger, W. C. (1936). Note concerning group influence upon Otis SA test scores. Journal of Educational Psychology, 27, 554. doi: 10.1037/h0057854 First citation in articleCrossref, Google Scholar

  • Luchins, A. S., & Luchins, E. H. (1979). Geometric problem solving related to differences in sex and mathematical interests. The Journal of Genetic Psychology, 134, 255–269. doi: 10.1080/00221325.1979.10534060 First citation in articleCrossref, Google Scholar

  • Mayer, A. (1904). Einzel- und Gesamtleistung des Schulkindes [Individual and general performance in the school child]. Leipzig, Germany: Barth. First citation in articleGoogle Scholar

  • Meumann, E. (1904). Haus- und Schularbeit [Homework and school work]. Leipzig, Germany: Barth. First citation in articleGoogle Scholar

  • Milgram, R. M., & Milgram, N. A. (1976). Group versus individual administration in the measurement of creative thinking in gifted and nongifted children. Child Development, 47, 563–565. doi: 10.2307/1128823 First citation in articleCrossref, Google Scholar

  • Moede, W. (1920). Experimentelle Massenpsychologie [Experimental psychology of the masses]. Leipzig, Germany: Hirzel. First citation in articleGoogle Scholar

  • Moore, H. T. (1917). Laboratory tests of anger, fear and sex interest. The American Journal of Psychology, 28, 390–395. doi: 10.2307/1413610 First citation in articleCrossref, Google Scholar

  • *Moser, R. S., Schatz, P., Neidzwski, K., & Ott, S. D. (2011). Group versus individual administration affects baseline neurocognitive test performance. The American Journal of Sports Medicine, 39, 2325–2330. doi: 10.1177/0363546511417114 First citation in articleCrossref, Google Scholar

  • *Neustifter, R. (2011). Einzel- vs. Gruppenvorgabe: Niveauunterschiede im Leistungsbereich [Individual- vs. group administration: Differences in the ability domain]. Wien, Austria: Universität Wien. First citation in articleGoogle Scholar

  • *Norris, R. C., Hottel, J. V., & Brooks, S. (1960). Comparability of Peabody Picture Vocabulary Test scores under group and individual administration. Journal of Educational Psychology, 51, 87–91. doi: 10.1037/h0046136 First citation in articleCrossref, Google Scholar

  • Pessin, J. (1933). The comparative effects of social and mechanical stimulation on memorizing. The American Journal of Psychology, 45, 263. doi: 10.2307/1414277 First citation in articleCrossref, Google Scholar

  • Preckel, F., & Thiemann, H. (2003). Online- versus paper-pencil-version of a high potential intelligence test. Swiss Journal of Psychology, 62, 131–138. doi: 10.1024/1421-0185.62.2.131 First citation in articleLink, Google Scholar

  • R Core Team. (2015). R: A language and environment for statistical computing. Wien, Austria: R Foundation for Statistical Computing. First citation in articleGoogle Scholar

  • *Reitman, W. R. (1960). Motivational induction and the behavior correlates of the achievement and affiliation motives. The Journal of Abnormal and Social Psychology, 60, 8. doi: 10.1037/h0042754 First citation in articleCrossref, Google Scholar

  • Rücker, G., Schwarzer, G., Carpenter, J. R., Binder, H., & Schumacher, M. (2011). Treatment-effect estimates adjusted for small-study effects via a limit meta-analysis. Biostatistics, 12, 122–142. doi: 10.1093/biostatistics/kxq046 First citation in articleCrossref, Google Scholar

  • *Shelton, J. T., Metzger, R. L., & Elliott, E. M. (2007). A group-administered lag task as a measure of working memory. Behavior Research Methods, 39, 482–493. doi: 10.3758/BF03193017 First citation in articleCrossref, Google Scholar

  • *Smith, W. J. (1952). Group effects on reasoning functions. Chicago, IL: Loyola University Chicago. First citation in articleGoogle Scholar

  • Sterne, J. A., Gavaghan, D., & Egger, M. (2000). Publication and related bias in meta-analysis: Power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology, 53, 1119–1129. doi: 10.1016/S0895-4356(00)00242-0 First citation in articleCrossref, Google Scholar

  • Strauss, B. (2002). Social facilitation in motor tasks: A review of research and theory. Psychology of Sport and Exercise, 3, 237–256. doi: 10.1016/S1469-0292(01)00019-X First citation in articleCrossref, Google Scholar

  • Triplett, N. (1898). The dynamogenic factors in pacemaking and competition. The American Journal of Psychology, 9, 507–533. doi: 10.2307/1412188 First citation in articleCrossref, Google Scholar

  • Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor Package. Journal of Statistical Software, 36, 1–84. doi: 10.18637/jss.v036.i03 First citation in articleCrossref, Google Scholar

  • *Weston, S. B., & English, H. B. (1926). The influence of the group on psychological test scores. The American Journal of Psychology, 37, 600–601. doi: 10.2307/1414922 First citation in articleCrossref, Google Scholar

  • Zajonc, R. B. (1965). Social facilitation. Science, 149, 269–274. doi: 10.1126/science.149.3681.269 First citation in articleCrossref, Google Scholar