Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article

Measurement Invariance of the Intelligence and Development Scales – 2 Across Language Versions, Gender, and Age

Published Online:June 06, 2023https://doi.org/10.1027/1015-5759/a000771

Abstract

Abstract: We examined the factor structure and measurement invariance of the intelligence and basic skills domains of the Intelligence and Development Scales – 2 (IDS-2) with the Dutch (N = 1,665) and German (N = 1,672) standardization samples. First, we tested five competing models using confirmatory factor analysis (CFA) on the Dutch data: two empirically based, derived earlier from the German data, and three theoretically based (IDS-2 and two Cattell–Horn–Carroll-based). Subsequently, we evaluated the measurement invariance of the final model across the Dutch and German versions and gender using multiple-group CFA and across age using local structural equation modeling. A second-order model with six first-order factors best represented the Dutch IDS-2 structure. Five IDS-2 factors were confirmed, but Visual Processing and Abstract Reasoning, and the intelligence and basic skills domains were not separable. This model displayed full invariance across the language versions and was largely invariant across gender and age (7–20 years). Thus, scores derived according to this model are comparable across these language versions, gender, and age. The strong general intelligence factor and weak broad ability factors ask for precaution when basing clinical interpretation on the broad ability factors.

References

  • Alloway, T. P., Gathercole, S. E., & Pickering, S. J. (2006). Verbal and visuospatial short-term and working memory in children: are they separable? Child Development, 77(6), 1698–1716. https://doi.org/10.1111/j.1467-8624.2006.00968.x First citation in articleCrossref, Google Scholar

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. First citation in articleGoogle Scholar

  • Baddeley, A. (2003). Working memory and language: An overview. Journal of Communication Disorders, 36(3), 189–208. https://doi.org/10.1016/S0021-9924(03)00019-4 First citation in articleCrossref, Google Scholar

  • Barrett, P. (2007). Structural equation modelling: Adjudging model fit. Personality and Individual Differences, 47(5), 815–824. https://doi.org/10.1016/j.paid.2006.09.018 First citation in articleCrossref, Google Scholar

  • Becker, M., Lüdtke, O., Trautwein, U., Köller, O., & Baumert, J. (2012). The differential effects of school tracking on psychometric intelligence: Do academic-track schools make students smarter? Journal of Educational Psychology, 104(3), 682–699. https://doi.org/10.1037/a0027608 First citation in articleCrossref, Google Scholar

  • Beaujean, A. A. (2015). John Carroll’s views on intelligence: Bi-factor vs. higher-order models. Journal of Intelligence, 3(4), 121–136. https://doi.org/10.3390/jintelligence3040121 First citation in articleCrossref, Google Scholar

  • Beaujean, A. A., & Benson, N. F. (2019). Theoretically-consistent cognitive ability test development and score interpretation. Contemporary School Psychology, 23, 126–137. https://doi.org/10.1007/s40688-018-0182-1 First citation in articleCrossref, Google Scholar

  • Bodovski, K., & Farkas, G. (2007). Mathematics growth in early elementary school: The roles of beginning knowledge, student engagement, and instruction. The Elementary School Journal, 108(2), 115–130. https://doi.org/10.1086/525550 First citation in articleCrossref, Google Scholar

  • Born, M. P., Bleichrodt, N., & van der Flier, H. (1987). Cross-cultural comparison of sex-related differences on intelligence tests: A meta-analysis. Journal of Cross-Cultural Psychology, 18(3), 283–314. https://doi.org/10.1177/0022002187018003002 First citation in articleCrossref, Google Scholar

  • Box, G. E. P. (1949). A general distribution theory for a class of likelihood criteria. Biometrika, 36(3/4), 317–346. https://doi.org/10.2307/2332671 First citation in articleCrossref, Google Scholar

  • Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Press. First citation in articleGoogle Scholar

  • Canivez, G. L., Grieder, S., & Bünger, A. (2021). Construct validity of the German Wechsler Intelligence Scale for Children, Fifth Edition: Exploratory and confirmatory factor analyses of the 15 primary and secondary subtests. Assessment, 28(2), 327–352. https://doi.org/10.1177/1073191120936330 First citation in articleCrossref, Google Scholar

  • Canivez, G. L., Watkins, M. W., & Dombrowski, S. C. (2017). Structural validity of the Wechsler Intelligence Scale for Children, Fifth Edition: Confirmatory factor analyses with the 16 primary and secondary subtests. Psychological Assessment, 29(4), 458–472. https://doi.org/10.1037/pas0000358 First citation in articleCrossref, Google Scholar

  • Canivez, G. L., & Youngstrom, E. A. (2019). Challenges to the Cattell-Horn-Carroll theory: Empirical, clinical, and policy implications. Applied Measurement in Education, 32(3), 232–248. https://doi.org/10.1080/08957347.2019.1619562 First citation in articleCrossref, Google Scholar

  • Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. Cambridge University Press. First citation in articleCrossref, Google Scholar

  • Cattell, R. B. (1941). Some theoretical issues in adult intelligence testing. Psychological Bulletin, 38, 592. First citation in articleGoogle Scholar

  • Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multi-Disciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834 First citation in articleCrossref, Google Scholar

  • Chen, F. F., Sousa, K. H., & West, S. G. (2005). Teacher’s corner: Testing measurement invariance of second-order factor models. Structural Equation Modeling, 12(3), 471–492. https://doi.org/10.1207/s15328007sem1203_7 First citation in articleCrossref, Google Scholar

  • Cucina, J. M., & Byle, K. (2017). The bifactor model fits better than the higher-order model in more than 90% of comparisons for mental abilities test batteries. Journal of Intelligence, 5(3), Article 27. https://doi.org/10.3390/jintelligence5030027 First citation in articleCrossref, Google Scholar

  • Cucina, J. M., & Howardson, G. N. (2017). Woodcock Johnson III, Kaufman Adolescent and Adult Intelligence Test (KAIT), Kaufman Assessment Battery for Children (KABC), and Differential Ability Scales (DAS) Support Carroll but Not Cattell-Horn. Psychological Assessment, 29(8), 1001–1015. https://doi.org/10.1037/pas0000389 First citation in articleCrossref, Google Scholar

  • DiStefano, C., & Dombrowski, S. C. (2006). Investigating the theoretical structure of the Stanford-Binet. Journal of Psychoeducational Assessment, 24(2), 123–136. https://doi.org/10.1177/0734282905285244 First citation in articleCrossref, Google Scholar

  • Eid, M., Geiser, C., Koch, T., & Heene, M. (2017). Anomalous results in G-factor models: Explanations and alternatives. Psychological Methods, 22(3), 541–562. https://doi.org/10.1037/met0000083 First citation in articleCrossref, Google Scholar

  • Eid, M., Krumm, S., Koch, T., & Schulze, J. (2018). Bifactor models for predicting criteria by general and specific factors: Problems of nonidentifiability and alternative solutions. Journal of Intelligence, 6(3), Article 42. https://doi.org/10.3390/jintelligence6030042 First citation in articleCrossref, Google Scholar

  • Facon, B., Magis, D., Nuchadee, M.-L., & De Boeck, P. (2011). Do Raven’s Colored Progressive Matrices function in the same way in typical and clinical populations? Insights from the intellectual disability field. Intelligence, 39, 281–291. https://doi.org/10.1016/j.intell.2011.04.002 First citation in articleCrossref, Google Scholar

  • Gignac, G. E. (2008). Higher-order models versus direct hierarchical models: G as superordinate or breadth factor? Psychology Science Quarterly, 50, 21–43. First citation in articleGoogle Scholar

  • Gignac, G. E., & Kretzschmar, A. (2017). Evaluating dimensional distinctness with correlated-factor models: Limitations and suggestions. Intelligence, 62, 138–147. https://doi.org/10.1016/j.intell.2017.04.001 First citation in articleCrossref, Google Scholar

  • Grieder, S., & Grob, A. (2020). Exploratory factor analyses of the Intelligence and Development Scales–2: Implications for theory and practice. Assessment, 27(8), 1853–1869. https://doi.org/10.1177/1073191119845051 First citation in articleCrossref, Google Scholar

  • Grieder, S. (2021). Structural and convergent validity of intelligence composites: Integrating evidence from three analysis levels [Doctoral dissertation]. Institutional Repository of the University of Basel. https://edoc.unibas.ch/83158/1/Dissertation_Grieder_Silvia_UBdef.pdf First citation in articleGoogle Scholar

  • Grieder, S., Visser, L., Timmerman, M., & Grob, A. (2022, December 8). Data and supplementary materials to “Measurement Invariance of the Intelligence and Development Scales – 2 Across Language Versions, Gender, and Age.”. https://osf.io/azep6 First citation in articleCrossref, Google Scholar

  • Grob, A., & Hagmann-von Arx, P. (2018). Intelligence and Development Scales 2 (IDS-2). Intelligenz- und Entwicklungsskalen für Kinder und Jugendliche. Manual zu Theorie, Interpretation und Gütekriterien [Intelligence and Development Scales for Children and Adolescents. Manual on theory, interpretation, and psychometric criteria]. Hogrefe. First citation in articleGoogle Scholar

  • Grob, A., Hagmann-von Arx, P., Ruiter, S. A. J., Timmerman, M. E., & Visser, L. (2018). Intelligence and Development Scales 2 (IDS-2). Intelligentie- en ontwikkelingsschalen voor kinderen en jongeren. Verantwoording en psychometrie [Intelligence and Development Scales for Children and Adolescents. Justification and psychometrics]. Hogrefe. First citation in articleGoogle Scholar

  • Hartung, J., Spormann, S. S., Moshagen, M., & Wilhelm, O. (2021). Structural differences in life satisfaction in a US adult sample across age. Journal of Personality, 89(6), 1232–1251. First citation in articleCrossref, Google Scholar

  • Hildebrandt, A., Lüdtke, O., Robitzsch, A., Sommer, C., & Wilhelm, O. (2016). Exploring factor model parameters across continuous variables with local structural equation models. Multivariate Behavioral Research, 51(2–3), 257–258. https://doi.org/10.1080/00273171.2016.1142856 First citation in articleCrossref, Google Scholar

  • Hildebrandt, A., Wilhelm, O., & Robitzsch, A. (2009). Complementary and competing factor analytic approaches for the investigation of measurement invariance. Review of Psychology, 16(2), 87–102. First citation in articleGoogle Scholar

  • Horn, J. L. (1991). Measurement of intellectual capabilities: A review of theory. In K. S. McGrewJ. WerderR. W. WoodcockEds., WJ-R Technical Manual (pp. 197–232). Riverside. First citation in articleGoogle Scholar

  • Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424–453. https://doi.org/10.1037/1082-989X.3.4.424 First citation in articleCrossref, Google Scholar

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118 First citation in articleCrossref, Google Scholar

  • Irwing, P. (2012). Sex differences in g: An analysis of the US standardization sample of the WAIS-III. Personality and Individual Differences, 53(2), 126–131. https://doi.org/10.1016/j.paid.2011.05.001 First citation in articleCrossref, Google Scholar

  • Kaufman, S. B., Reynolds, M. R., Liu, X., Kaufman, A. S., & McGrew, K. S. (2012). Are cognitive g and academic achievement g one and the same g? An Exploration on the Woodcock-Johnson and Kaufman Tests. Intelligence, 40(2), 123–138. https://doi.org/10.1016/j.intell.2012.01.009 First citation in articleCrossref, Google Scholar

  • Keith, T. Z., & Reynolds, M. R. (2010). Cattell-Horn-Carroll abilities and cognitive tests: What we’ve learned from 20 years of research. Psychology in the Schools, 47(7), 635–650. https://doi.org/10.1002/pits.20496 First citation in articleCrossref, Google Scholar

  • MacLean, H., McKenzie, K., Kidd, J., Murray, A. L., & Schwannauer, M. (2011). Measurement invariance in the assessment of people with an intellectual disability. Research in Developmental Disabilities, 32, 1081–1085. https://doi.org/10.1016/j.ridd.2011.01.022 First citation in articleCrossref, Google Scholar

  • Marsh, H. W., Hau, K.-T., Balla, J. R., & Grayson, D. (1998). Is more ever too much? The number of indicators per factor in confirmatory factor analysis. Multivariate Behavioral Research, 33(2), 181–220. https://doi.org/10.1207/s15327906mbr3302_1 First citation in articleCrossref, Google Scholar

  • McDonald, R. P. (1985). Factor analysis and related methods. Erlbaum. First citation in articleGoogle Scholar

  • McDonald, R. P. (1999). Test theory: A unified treatment. Taylor & Francis. First citation in articleGoogle Scholar

  • McGrew, K. S. (1997). Analysis of the major intelligence batteries according to a proposed comprehensive gf-gc framework. In D. P. FlanaganJ. L. GenshaftP. L. HarrisonEds., Contemporary Intellectual Assessment: Theories, Tests, and Issues (pp. 151–179). Guilford Press. First citation in articleGoogle Scholar

  • Naglieri, J. A. (2014). Hundred years of intelligence testing: Moving from traditional IQ to second-generation intelligence tests. In S. GoldsteinD. PrinciottaJ. A. NaglieriEds., Handbook of intelligence: Evolutionary theory, historical perspective, and current concepts (pp. 295–316). Springer. https://doi.org/10.1007/978-1-4939-1562-0_20 First citation in articleCrossref, Google Scholar

  • Nel, D. G., & van der Merwe, C. A. (1986). A solution to the multivariate Behrens-Fisher problem. Communications in Statistics-Theory and Methods, 15(12), 3719–3735. https://doi.org/10.1080/03610928608829342 First citation in articleCrossref, Google Scholar

  • Pokropek, A., Marks, G. N., & Borgonovi, F. (2021). How much do students’ scores in PISA reflect general intelligence and how much do they reflect specific abilities? Journal of Educational Psychology. Advance online publication. https://doi.org/10.1037/edu0000687 First citation in articleGoogle Scholar

  • Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. https://doi.org/10.1080/00273171.2012.715555 First citation in articleCrossref, Google Scholar

  • Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2), 129–140. https://doi.org/10.1080/00223891.2012.725437 First citation in articleCrossref, Google Scholar

  • Robitzsch, A. (2020). Sirt: Supplementary item response theory models. R Package Version 3.10.-31. https://CRAN.R-project.org/package=sirt First citation in articleGoogle Scholar

  • Roivainen, E. (2011). Gender differences in processing speed: A review of recent research. Learning and Individual Differences, 21(2), 145–149. https://doi.org/10.1016/j.lindif.2010.11.021 First citation in articleCrossref, Google Scholar

  • Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02 First citation in articleCrossref, Google Scholar

  • Schmid, J., & Leiman, J. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22, 53–61. https://doi.org/10.1007/BF02289209 First citation in articleCrossref, Google Scholar

  • Schmitt, T. A. (2011). Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psychoeducational Assessment, 29(4), 304–321. https://doi.org/10.1177/0734282911406653 First citation in articleCrossref, Google Scholar

  • Schneider, W. J., & McGrew, K. S. (2018). The Cattell-Horn-Carroll theory of cognitive abilities. In D. P. FlanaganE. M. McDonoughEds., Contemporary intellectual assessment: Theories, tests, and issues (pp. 73–163). Guilford Press. First citation in articleGoogle Scholar

  • Schweizer, F., Hagmann-von Arx, P., Ledermann, T., & Grob, A. (2018). Geschlechtsinvarianz und Geschlechtsdifferenzen in der Intelligenzeinschätzung mit den Intelligence and Development Scales [Sex invariance and sex differences in intelligence assessments with the Intelligence and Development Scales]. Diagnostica, 64(4), 203–214. https://doi.org/10.1026/0012-1924/a000207 First citation in articleLink, Google Scholar

  • Steenkamp, J.-B. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78–90. https://doi.org/10.1086/209528 First citation in articleCrossref, Google Scholar

  • Steiner, M., & Grieder, S. (2020). EFAtools: An R package with fast and flexible implementations of exploratory factor analysis tools. Journal of Open Source Software, 5(53), Article 2521. https://doi.org/10.21105/joss.02521 First citation in articleCrossref, Google Scholar

  • Tarchi, C. (2010). Reading comprehension of informative texts in secondary school: A focus on direct and indirect effects of reader’s prior knowledge. Learning and Individual Differences, 20(5), 415–420. https://doi.org/10.1016/j.lindif.2010.04.002 First citation in articleCrossref, Google Scholar

  • Tucker-Drob, E. M. (2009). Differentiation of cognitive abilities across the lifespan. Developmental Psychology, 45(4), 1097–1118. https://doi.org/10.1037/a0015864 First citation in articleCrossref, Google Scholar

  • Voyer, D., Voyer, S., & Bryden, M. P. (1995). Magnitude of sex differences in spatial abilities: A meta-analysis and consideration of critical variables. Psychological Bulletin, 117(2), 250–270. https://doi.org/10.1037/0033-2909.117.2.250 First citation in articleCrossref, Google Scholar

  • Wasserman, J. D. (2019). Deconstructing CHC. Applied Measurement in Education, 32(3), 249–268. https://doi.org/10.1080/08957347.2019.1619563 First citation in articleCrossref, Google Scholar