Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Articles

Representation of Competencies in Multidimensional IRT Models with Within-Item and Between-Item Multidimensionality

Published Online:February 26, 2015https://doi.org/10.1027/0044-3409.216.2.89

Abstract

Multidimensional item response theory (MIRT) holds considerable promise for the development of psychometric models of competence. It provides an ideal foundation for modeling performance in complex domains, simultaneously taking into account multiple basic abilities. The aim of this paper is to illustrate the relations between a two-dimensional IRT model with between-item multidimensionality and a nested-factor model with within-item multidimensionality, and the different substantive meanings of the ability dimensions in the two models. Both models are applied to empirical data from a large-scale assessment of reading and listening comprehension in a foreign language. In the between-item model, performance in the reading and listening items is modeled by two separate dimensions. In the within-item model, one dimension represents the abilities common to both tests, and a second dimension represents abilities specific to listening comprehension. Distinct relations of external variables, such as gender and cognitive abilities, with ability scores demonstrate that the alternative models have substantively different implications.

References

  • Adams, R. , Wilson, M. , Wang, W.-C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1–23. First citation in articleCrossref, Google Scholar

  • Beck, B. , Bundt, S. , Gomolka, J. (2008). Ziele und Anlage der DESI-Studie [Aims and layout of the DESI study]. In E. Klieme, W. Eichler, A. Helmke, R.H. Lehmann, G. Nold, H.-G. Rolff et al. (Eds.), Unterricht und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESI-Studie [Instruction and competence development in German and English. Results of the DESI study] (pp. 11–33). Weinheim: Beltz. First citation in articleGoogle Scholar

  • Beck, B. , Klieme, E. (Eds.). (2007). Sprachliche Kompetenzen. Konzepte und Messung [Language competencies. Concepts and measurement]. Weinheim: Beltz. First citation in articleGoogle Scholar

  • Bollen, K.A. (1989). Structural equations with latent variables. New York: Wiley. First citation in articleCrossref, Google Scholar

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Embretson, S.E. (2006). The continued search for nonarbitrary metrics in psychology. American Psychologist, 61, 50–55. First citation in articleCrossref, Google Scholar

  • Fay, R.E. (1989). Theory and application of replicate weighting for variance calculations. In Proceedings of the Section on Survey Research Methods (pp. 212–217). Washington, DC: American Statistical Association. First citation in articleGoogle Scholar

  • Hartig, J. , Jude, N. (2008). Sprachkompetenzen von Mädchen und Jungen [Language competencies of girls and boys]. In E. Klieme, W. Eichler, A. Helmke, R.H. Lehmann, G. Nold, H.-G. Rolff et al. (Eds.), Unterricht und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESI-Studie [Instruction and competence development in German and English. Results of the DESI study] (pp. 202–207). Weinheim: Beltz. First citation in articleGoogle Scholar

  • Hartig, J. , Jude, N. , Wagner, W. (2008). Methodische Grundlagen der Messung und Erklärung sprachlicher Kompetenzen [Methods and basic principles of the measurement and prediction of language competencies]. In E. Klieme, W. Eichler, A. Helmke, R.H. Lehmann, G. Nold, H.-G. Rolff et al. (Eds.), Unterricht und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESI-Studie [Instruction and competence development in German and English. Results of the DESI study] (pp. 34–54). Weinheim: Beltz. First citation in articleGoogle Scholar

  • Heller, K.A. , Perleth, C. (2000). Kognitiver Fähigkeits-Test (Rev.) für 5.–12. Klassen [Cognitive ability test – revised – for the 5th to 12th grade]. Göttingen: Beltz. First citation in articleGoogle Scholar

  • Klieme, E. , Eichler, W. , Helmke, A. , Lehmann, R.H. , Nold, G. , Rolff, H.-G. et al. (2008). Unterricht und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESI-Studie [Instruction and competence development in German and English. Results of the DESI study]. Weinheim: Beltz. First citation in articleGoogle Scholar

  • Koeppen, K. , Hartig, J. , Klieme, E. , Leutner, E. (2008). Current issues in research on competence modeling and assessment. Zeitschrift für Psychologie / Journal of Psychology, 216, 60–72. First citation in articleLink, Google Scholar

  • McDonald, R.P. (1997). Normal-ogive multidimensional model. In W.J. van der Linden & R.K. Hambleton (Eds.), Handbook of modern item response theory (pp. 257–269). New York: Springer-Verlag. First citation in articleCrossref, Google Scholar

  • McDonald, R.P. (2000). A basis for multidimensional item response theory. Applied Psychological Measurement, 24, 99–114. First citation in articleCrossref, Google Scholar

  • Mislevy, R.J. , Beaton, A.E. , Kaplan, B. , Sheehan, K.M. (1992). Estimating population characteristics from sparse matrix samples of responses. Journal of Educational Measurement, 29, 133–161. First citation in articleCrossref, Google Scholar

  • Mulaik, S.A. , Quartetti, D.A. (1997). First order or higher order general factors? Structural Equation Modeling, 4, 193–211. First citation in articleCrossref, Google Scholar

  • Muthén, B.O. (2002). Beyond SEM: General latent variable modelling. Behaviormetrika, 29, 81–117. First citation in articleCrossref, Google Scholar

  • Muthén, L.K. , Muthén, B.O. (2007). Mplus user’s guide (4th ed.). Los Angeles, CA: Muthén & Muthén. First citation in articleGoogle Scholar

  • Nold, G. , Rossa, H. (2007a). Hörverstehen [Listening comprehension]. In B. Beck, E. Klieme (Eds.), Sprachliche Kompetenzen: Konzepte und Messung [Language competencies: Concepts and measurement] (pp. 178–196). Weinheim: Beltz. First citation in articleGoogle Scholar

  • Nold, G. , Rossa, H. (2007b). Leseverstehen [Reading comprehension]. In B. Beck, E. Klieme (Eds.), Sprachliche Kompetenzen. Konzepte und Messung [Language competencies: Concepts and measurement] (pp. 197–211). Weinheim: Beltz. First citation in articleGoogle Scholar

  • Nold, G. , Rossa, H. (2008a). Hörverstehen Englisch [English listening comprehension]. In E. Klieme, W. Eichler, A. Helmke, R.H. Lehmann, G. Nold, H.-G. Rolff et al. (Eds.), Unterricht und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESI-Studie [Instruction and competence development in German and English. Results of the DESI study] (pp. 120–129). Weinheim: Beltz. First citation in articleGoogle Scholar

  • Nold, G. , Rossa, H. (2008b). Sprechen English [Spoken English]. In E. Klieme, W. Eichler, A. Helmke, R.H. Lehmann, G. Nold, H.-G. Rolff et al. (Eds.), Unterricht und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESI-Studie [Instruction and competence development in German and English. Results of the DESI study] (pp. 170–179). Weinheim: Beltz. First citation in articleGoogle Scholar

  • Nold, G. , Rossa, H. , Chatzivassiliadou, K. (2008). Leseverstehen Englisch [English reading comprehension]. In E. Klieme, W. Eichler, A. Helmke, R.H. Lehmann, G. Nold, H.-G. Rolff et al. (Eds.), Unterricht und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESI-Studie [Instruction and competence development in German and English. Results of the DESI study] (pp. 130–138). Weinheim: Beltz. First citation in articleGoogle Scholar

  • Ordinate . (2004). SET-10 ® Test description. Validation summary. Menlo Park, CA: Ordinate. First citation in articleGoogle Scholar

  • Rabe-Hesketh, S. , Skrondal, A. , Pickles, A. (2004). GLAMM manual. UC Berkeley Division of Biostatistics Working Paper No. 160. First citation in articleGoogle Scholar

  • Reckase, M.D. (1997a). A linear logistic multidimensional model for dichotomous item response data. In W.J. van der Linden & R.K. Hambleton (Eds.), Handbook of modern item response theory (pp. 271–286). New York: Springer-Verlag. First citation in articleCrossref, Google Scholar

  • Reckase, M.D. (1997b). The past and future of multidimensional item response theory. Applied Psychological Measurement, 21, 25–36. First citation in articleCrossref, Google Scholar

  • Skrondal, A. , Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models. Boca Raton: Chapman & Hall. First citation in articleCrossref, Google Scholar

  • Walker, C.M. , Beretvas, S.N. (2003). Comparing multidimensional and unidimensional proficiency classifications: Multidimensional IRT as a diagnostic aid. Journal of Educational Measurement, 40, 255–275. First citation in articleCrossref, Google Scholar

  • Westat . (2000). WesVar 4.0 user’s guide. Rockville, MD: Author. First citation in articleGoogle Scholar

  • Wu, M. , Adams, R. , Wilson, M. (1998). ConQuest: Generalized item response modeling software. Melbourne: Australian Council for Educational Research. First citation in articleGoogle Scholar