Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Originalia

Wie valide sind Papier-und-Bleistift-Tests zur Erfassung computerbezogener Kompetenzen?

Published Online:January 09, 2014https://doi.org/10.1026/0012-1924/a000114

Abstract

Ziel dieser Untersuchung war es, einen Papier-und-Bleistift-Test zur Erfassung von computerbezogenen Kompetenzen mit einem computerbasierten Performanztest hinsichtlich spezifischer Validitätsaspekte zu vergleichen. Der Papier-und-Bleistift-Test mit 36 Items wurde im Rahmen der National Educational Panel Study (NEPS) für Jugendliche am Ende der Sekundarstufe I entwickelt und an 396 Jugendlichen hinsichtlich seiner psychometrischen Qualität überprüft. Für die Validierung wurde einer Teilstichprobe (n = 133) in einem Within-Persons-Design eine Hälfte des Tests in Form computerbasierter Performanzitems vorgegeben, die andere Hälfte in der ursprünglichen Form, d. h. als Papier-und-Bleistift-Test präsentiert. Neben der Struktur der Zusammenhänge beider Testformen wurden die konvergente und diskriminante Validität beider Testmodi über Zusammenhänge mit theoretisch einschlägigen Referenzkonstrukten (kognitive Grundfähigkeiten, computerbezogene Schülermerkmale) überprüft. Die Ergebnisse weisen für beide Testformen auf eine ausreichende Validität hin. Jedoch ergeben sich für den computerbasierten Performanztest hinsichtlich aller Kriteriumsvariablen signifikant bessere Kennwerte.


How valid are paper-and-pencil-based tests to measure computer-related competencies?

The aim of this study was to evaluate a paper-pencil test measuring computer-related competencies compared to a computer-based performance test. The paper-pencil test consisted of 36 items and was developed in the context of the German National Educational Panel Study (NEPS). Its psychometric quality was reviewed using a sample of 396 students from secondary school. In a within-persons design one half of the items were presented in a computer-based performance test to a subsample of 133; students the other half of the items were presented in a paper-pencil test to the same subsample. The convergent and discriminant validity of both test forms was reviewed by comparing the correlations with theoretically appropriate reference constructs (basic cognitive skills, computer-related student characteristics). Although the results indicate an adequate validity for both test forms, the computer-based performance test showed significantly better parameters regarding all criterion variables.

Literatur

  • Baumert, J. , Brunner, M. , Lüdtke, O. & Trautwein, U. (2007). Was messen internationale Schulleistungsstudien?–Resultate kumulativer Wissenerwerbsprozesse. Psychologische Rundschau, 58, 118 – 145. First citation in articleLink, Google Scholar

  • Blossfeld, H.-P. , Doll, J. & Schneider, T. (2008). Bildungsprozesse im Lebenslauf: Grundzüge der zukünftigen Bildungspanelstudie für die Bundesrepublik Deutschland. Recht der Jugend und des Bildungswesens, 56, 321 – 328. First citation in articleCrossref, Google Scholar

  • ETS [Educational Testing Service]. (2002). Digital transformation. A framework for ICT literacy. Princeton, NJ: ETS. First citation in articleGoogle Scholar

  • Ezziane, Z. (2007). Information technology literacy: Implications on teaching and learning. Educational Technology & Society, 10, 175 – 191. First citation in articleGoogle Scholar

  • Goldhammer, F. , Naumann, J , & Keßel, Y. (2013). Assessing individual differences in basic computer skills: Psychometric characteristics of an interactive performance measure. European Journal of Psychological Assessment, 29, 263 – 275. doi: 10.1027/1015 – 5759/a000153. First citation in articleLink, Google Scholar

  • Heller, K. A. & Perleth, C. (2000). Kognitiver Fähigkeitstest für 4.–12. Klassen, Revision (KFT 4 – 12+ R). Göttingen: Hogrefe. First citation in articleGoogle Scholar

  • Huff, K. L. & Sireci, S. G. (2001). Validity issues in computer-based testing. Educational Measurement: Issues and Practice, 20, 16 – 25. First citation in articleCrossref, Google Scholar

  • Katz, I. R. (2007). Testing information literacy in digital environments: The ETS iskillsTM assessment. Information Technology and Libraries, 26, 3 – 12. First citation in articleCrossref, Google Scholar

  • Klieme, E. (2004). Was sind Kompetenzen und wie lassen sie sich messen? Pädagogik, 6, 10 – 13. First citation in articleGoogle Scholar

  • Kuhlemeier, H. & Hemker, B. (2007). The impact of computer use at home at home on students’ Internet skills. Computers & Education, 49, 460 – 480. First citation in articleCrossref, Google Scholar

  • Leu, D. J. , Kinzer, C. K. , Coiro, J. L. & Cammack, D. W. (2004). Toward a theory of new literacies emerging from the internet and other information and communication technologies. In N. J. Unrau & R. B. Russell (Eds.), Theoretical models and processes of reading (pp. 1570 – 1613). Newark, NJ: International Reading Association. First citation in articleGoogle Scholar

  • Markauskaite, L. (2007). Exploring the structure of trainee teachers‘ ICT literacy: The main components of, and relationships between, general cognitive and technical capabilities. Educational Technology Research and Development, 55, 547 – 572 First citation in articleCrossref, Google Scholar

  • Mayer, R. E. (2002). A taxonomy for computer-based assessment of problem solving. Computers in Human Behavior, 18, 623 – 632. First citation in articleCrossref, Google Scholar

  • Medienpädagogischer Forschungsverbund Südwest (2011). JIM-Studie 2011. Jugend, Information, (Multi ‐)Media. Stuttgart: Medienpädagogischer Forschungsverbund Südwest. First citation in articleGoogle Scholar

  • Ministerial Council on Education, Employment , Training and Youth Affairs (MCEETYA). (2007). National Assessment Program. Information and communication technology literacy 2005. Years 6 and 10. Melbourne, Australia: MCEETYA. First citation in articleGoogle Scholar

  • Muthen, L. K. & Muthen, B. O. (2009). Mplus user‘s guide. Los Angeles, CA: Muthen & Muthen. First citation in articleGoogle Scholar

  • Potosky, D. (2007). The internet knowledge (iKnow) measure. Computers in Human Behavior, 23, 2670 – 2777. First citation in articleCrossref, Google Scholar

  • Poynton, T. A. (2005). Computer literacy across the lifespan: A review with implications for educators. Computers in Human Behavior, 21, 861 – 872. First citation in articleCrossref, Google Scholar

  • Quellmalz, E. S. & Kozma, R. (2003). Designing assessments of learning with technology. Assessment in Education, 10, 389 – 407. First citation in articleCrossref, Google Scholar

  • Richter, T. , Naumann, J. & Horz, H. (2010). Eine revidierte Fassung des Inventars zur Computerbildung (INCOBI-R). Zeitschrift für Pädagogische Psychologie, 24, 23 – 37. First citation in articleLink, Google Scholar

  • Senkbeil, M. , Ihme, J. M. & Wittwer, J. (in Druck). Entwicklung und erste Validierung eines Tests zur Erfassung technologischer und informationsbezogener Literacy (TILT) für Jugendliche am Ende der Sekundarstufe I. Zeitschrift für Erziehungswissenschaft. First citation in articleGoogle Scholar

  • Sireci, S. G. & Zenisky, A. L. (2006). Innovative item formats in computer-based testing: In pursuit of improved construct representation. In S. M. Downing & T. M. Haladyna (Eds.), Handbook of test development (pp. 329 – 347). First citation in articleGoogle Scholar

  • Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245 – 251. First citation in articleCrossref, Google Scholar

  • Süß, H.-M. (1996). Intelligenz, Wissen und Problemlösen. Göttingen: Hogrefe. First citation in articleGoogle Scholar

  • Weinert, S. , Artelt, C. Prenzel , M., Senkbeil , M., Ehmke , T. & Carstensen, C. H. (2011). Development of competencies across the life span. Zeitschrift für Erziehungswissenschaft, 14 (Suppl. 2), 67 – 86. First citation in articleCrossref, Google Scholar

  • Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Wittwer, J. , & Senkbeil, M. (2008). Is students’ computer use at home related to their mathematical performance at school? Computers & Education, 50, 1558 – 1571. First citation in articleGoogle Scholar

  • Wu, M. L. , Adams, R. J. , Wilson, M. R. & Haldane, S. (2007). ACER ConQuest 2.0–Generalised item response modeling software. Camberwell: ACER Press. First citation in articleGoogle Scholar