Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Open Access

Effects of Test Mode and Medium on Elementary School Students’ Test Experience

Published Online:May 23, 2023https://doi.org/10.1027/1015-5759/a000767

Abstract

Abstract: The use of digital media in education can bring great benefits and its use in schooling is steadily increasing. Administrating paper- versus computer-based as well as fixed-item versus adaptive tests could create differences in test experience, which can threaten the comparability of test results. This study investigated how the pen-and-paper, computer-based, and computer-adaptive test formats of a standardized reading comprehension test affect test anxiety and motivation among German fourth-grade students. A within-class randomized field trial with 387 fourth graders (aged: 9–10 years; 46.3% female) was conducted. Repeated-measures analysis of covariance (ANCOVA) revealed no differences in state test anxiety between the test formats when controlling for trait test anxiety and pre-test state anxiety, but state reading motivation was initially higher when reading on a screen, controlling for trait reading motivation. However, this difference diminishes over the course of the test. Implications for using digital media in elementary school test situations are discussed.

References

  • Baumert, J., & Demmrich, A. (2001). Test motivation in the assessment of student skills: The effects of incentives on motivation and performance. European Journal of Psychology of Education, 16(3), 441–462. https://doi.org/10.1007/BF03173192 First citation in articleCrossref, Google Scholar

  • Bertrams, A., & Englert, C. (2014). Test anxiety, self-control, and knowledge retrieval in secondary school students. Zeitschrift Für Entwicklungspsychologie Und Pädagogische Psychologie, 46(4), 165–170. https://doi.org/10.1026/0049-8637/a000111 First citation in articleLink, Google Scholar

  • Brüggemann, T. (2023). Supplement to “Effects of test mode and medium on elementary school students’ test experience” https://osf.io/76hc2/?view_only=baf5985b8cb94fe4b78cbce4261aee7a. https://doi.org/10.17605/OSF.IO/76HC2 First citation in articleGoogle Scholar

  • Chua, Y. P. (2012). Effects of computer-based testing on test performance and testing motivation. Computers in Human Behavior, 28(5), 1580–1586. https://doi.org/10.1016/j.chb.2012.03.020 First citation in articleCrossref, Google Scholar

  • Colwell, N. M. (2013). Test anxiety, computer-adaptive testing, and the common core. Journal of Education and Training Studies, 1(2), 50–60. https://doi.org/10.11114/jets.vli2.101 First citation in articleCrossref, Google Scholar

  • dos Santos, T. D., & de Santana, V. F. (2018). Computer anxiety and interaction: A systematic review. In Proceedings of the 15th International Web for All Conference, (pp. 1–10). https://doi.org/10.1145/3192714.3192825 First citation in articleCrossref, Google Scholar

  • Englert, C., Bertrams, A., & Dickhäuser, O. (2011). Entwicklung der Fünf-Item Kurzskala STAI-SKD zur Messung von Zustandsangst [Development of the five-item short scale STAI-SKD for the assessment of state anxiety]. Zeitschrift für Gesundheitspsychologie, 19, 173–180. https://doi.org/10.1026/0943-8149/a000049 First citation in articleLink, Google Scholar

  • Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2020). Preparing for life in a digital world: IEA international computer and information literacy study 2018 international report. Springer Nature. https://doi.org/10.1007/978-3-030-38781-5 First citation in articleCrossref, Google Scholar

  • Frey, A., Hartig, J., & Moosbrugger, H. (2009). Effekte des adaptiven Testens auf die Motivation zur Testbearbeitung am Beispiel des Frankfurter Adaptiven Konzentrationsleistungs-Tests [Effects of adaptive testing on test-taking motivation with the example of the Frankfurt Adaptive Concentration Test]. Diagnostica, 55(1), 20–28. https://doi.org/10.1026/0012-1924.55.1.20 First citation in articleLink, Google Scholar

  • Helm, C., & Warwas, J. (2018). Psychological determinants of test motivation in low-stakes test situations: A longitudinal study of singletrait–multistate models in accounting. Empirical Research in Vocational Education and Training, 10(1), 1–34. https://doi.org/10.1186/s40461-018-0074-7 First citation in articleCrossref, Google Scholar

  • Hußmann, A., Wendt, H., Bos, W., Bremerich-Vos, A., Kasper, D., Lankes, E.-M., McElvany, N., Stubbe, T. C., & Valtin, R. (2017). IGLU 2016. Lesekompetenzen von Grundschulkindern in Deutschland im internationalen Vergleich [International comparison of German elementary school students’ reading competencies]. Waxmann. First citation in articleGoogle Scholar

  • Kunter, M., Schümer, G., Artelt, C., Baumert, J., Klieme, E., Neubrand, M., Prenzel, M., Schiefele, U., Schneider, W., Stanat, P., Tillman, K.-J., & Weiß, M. (2002). PISA 2000: Dokumentation der Erhebungsinstrumente (Materialien aus der Bildungsforschung Nr. 72) [PISA 2000: Documentation of the survey instruments (Educational research materials No. 72)]. Max-Planck-Institut für Bildungsforschung. First citation in articleGoogle Scholar

  • Lepper, C., Stang, J., & McElvany, N. (2021). Gender differences in text‐based interest: Text characteristics as underlying variables. Reading Research Quarterly, 57(2), 537–554. https://doi.org/10.1002/rrq.420 First citation in articleCrossref, Google Scholar

  • Ling, G., Attali, Y., Finn, B., & Stone, E. A. (2017). Is a computerized adaptive test more motivating than a fixed-item test? Applied Psychological Measurement, 41(7), 495–511. https://doi.org/10.1177/0146621617707556 First citation in articleCrossref, Google Scholar

  • Ludewig, U., Trendtel, M., Schlitter, T., & McElvany, N. (2021). Adaptives Testen von Textverständnis in der Grundschule [Adaptive Testing of Text Comprehension in Primary School. Development of a CAT-Optimized Item Pool]. Diagnostica, 68(1), 39–50. First citation in articleLink, Google Scholar

  • Magis, D., & Raîche, G. (2012). Random generation of response patterns under computerized adaptive testing with the R package catR. Journal of Statistical Software, 48(8), 1–31. https://doi.org/10.18637/jss.v048.i08 First citation in articleCrossref, Google Scholar

  • Martin, A. J., & Lazendic, G. (2018). Computer-adaptive testing: Implications for students’ achievement, motivation, engagement, and subjective test experience. Journal of Educational Psychology, 110(1), 27–45. https://doi.org/10.1037/edu0000205 First citation in articleCrossref, Google Scholar

  • Ortner, T. M., & Caspers, J. (2011). Consequences of test anxiety on adaptive versus fixed item testing. European Journal of Psychological Assessment, 27(3), 157–163. https://doi.org/10.1027/1015-5759/a000062 First citation in articleLink, Google Scholar

  • Ortner, T. M., Weißkopf, E., & Koch, T. (2014). I will probably fail. European Journal of Psychological Assessment, 30(1), 48–56. https://doi.org/10.1027/1015-5759/a000168 First citation in articleLink, Google Scholar

  • Paulman, R. G., & Kennelly, K. J. (1984). Test anxiety and ineffective test taking: Different names, same construct? Journal of Educational Psychology, 76(2), 279–288. https://doi.org/10.1037/0022-0663.76.2.279 First citation in articleCrossref, Google Scholar

  • Picton, I. (2014). The impact of ebooks on the reading motivation and reading skills of children and young people: A rapid literature review. National Literacy Trust. Retrieved October 13, 2022 from https://eric.ed.gov/?id=ed560635 First citation in articleGoogle Scholar

  • R Core Team. (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing. First citation in articleGoogle Scholar

  • Sahlan, F., Alberth, Madil, W., & Hutnisyawati (2021). The effects of modes of test administration on test anxiety and test scores: A study in an Indonesian school. Issues in Educational Research, 31(3), 952–971. https://www.iier.org.au/iier31/sahlan.pdf First citation in articleGoogle Scholar

  • Schwabe, F., McElvany, N., & Trendtel, M. (2015). The school age gender gap in reading achievement: Examining the influences of item format and intrinsic reading motivation. Reading Research Quarterly, 50(2), 219–232. https://doi.org/10.1002/rrq.92 First citation in articleCrossref, Google Scholar

  • Shermis, M. D., & Lombard, D. (1998). Effects of computer-based test administrations on test anxiety and performance. Computers in Human Behavior, 14(1), 111–123. https://doi.org/10.1016/S0747-5632(97)00035-6 First citation in articleCrossref, Google Scholar

  • Shin, G., Feng, Y., Jarrahi, M. H., & Gafinowitz, N. (2019). Beyond novelty effect: A mixed-methods exploration into the motivation for long-term activity tracker use. JAMIA Open, 2(1), 62–72. https://doi.org/10.1093/jamiaopen/ooy048 First citation in articleCrossref, Google Scholar

  • Sieber, J. E., O’Neil, J., & Tobias, S. (1977). Anxiety, learning, and instruction. Routledge. https://doi.org/10.4324/9780203056684 First citation in articleGoogle Scholar

  • Tonidandel, S., Quiñones, M. A., & Adams, A. A. (2002). Computer-adaptive testing: The impact of test characteristics on perceived performance and test takers’ reactions. Journal of Applied Psychology, 87(2), 320–332. https://doi.org/10.1037/0021-9010.87.2.320 First citation in articleCrossref, Google Scholar

  • Tremblay, P. F., Goldberg, M. P., & Gardner, R. C. (1995). Trait and state motivation and the acquisition of Hebrew vocabulary. Canadian Journal of Behavioural Science/Revue Canadienne des Sciences du Comportement, 27(3), 356–370. https://doi.org/10.1037/0008-400X.27.3.356 First citation in articleCrossref, Google Scholar

  • von der Embse, N., Jester, D., Roy, D., & Post, J. (2018). Test anxiety effects, predictors, and correlates: A 30-year meta-analytic review. Journal of Affective Disorders, 227, 483–493. https://doi.org/10.1016/j.jad.2017.11.048 First citation in articleCrossref, Google Scholar

  • Wacker, A., Jaunzeme, J., & Jaksztat, S. (2008). Eine Kurzform des Prüfungsängstlichkeitsinventars TAI-G [A short version of the Test Anxiety Inventory TAI-G]. Zeitschrift für Pädagogische Psychologie, 22(1), 73–81. https://doi.org/10.1024/1010-0652.22.1.73 First citation in articleLink, Google Scholar

  • Warm, T. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54(3), 427–450. https://doi.org/10.1007/bf02294627 First citation in articleCrossref, Google Scholar

  • Weiss, D. J., & Betz, N. E. (1973). Ability measurement: Conventional or adaptive? (Research Report 73-1). Department of Psychology, University of Minnesota. First citation in articleGoogle Scholar

  • Wise, S. L., & DeMars, C. E. (2006). An application of item response time: The effort-moderated IRT model. Journal of Educational Measurement, 43(1), 19–38. https://doi.org/10.1111/j.1745-3984.2006.00002.x First citation in articleCrossref, Google Scholar

  • Yamamoto, K., Shin, H., & Khorramdel, L. (2019). Introduction of multistage adaptive testing design in PISA 2018 (OECD Education Working Papers No. 209). OECD Publishing. https://doi.org/10.1787/b9435d4b-en First citation in articleGoogle Scholar

  • Zohar, D. (1998). An additive model of test anxiety: Role of exam-specific expectations. Journal of Educational Psychology, 90(2), 330–340. https://doi.org/10.1037/0022-0663.90.2.330 First citation in articleCrossref, Google Scholar