Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Open AccessOriginal Article

How Representational Pictures Enhance Students’ Performance and Test-Taking Pleasure in Low-Stakes Assessment

Published Online:October 07, 2016https://doi.org/10.1027/1015-5759/a000351

Abstract

Abstract. Pictures are often used in standardized educational large-scale assessment (LSA), but their impact on test parameters has received little attention up until now. Even less is known about pictures’ affective effects on students in testing (i.e., test-taking pleasure and motivation). However, such knowledge is crucial for a focused application of multiple representations in LSA. Therefore, this study investigated how adding representational pictures (RPs) to text-based item stems affects (1) item difficulty and (2) students’ test-taking pleasure. An experimental study with N = 305 schoolchildren was conducted, using 48 manipulated parallel science items (text-only vs. text-picture) in a rotated multimatrix design to realize within-subject measures. Students’ general cognitive abilities, reading abilities, and background variables were assessed to consider potential interactions between RPs’ effects and students’ performance. Students also rated their item-solving pleasure for each item. Results from item-response theory (IRT) model comparisons showed that RPs only reduced item difficulty when pictures visualized information mandatory for solving the task, while RPs substantially enhanced students’ test-taking pleasure even when they visualized optional context information. Overall, our findings suggest that RPs have a positive cognitive and affective influence on students’ performance in LSA (i.e., multimedia effect in testing) and should be considered more frequently.

References

  • Ainsworth, S. (1999). The functions of multiple representations. Computers & Education, 33, 131–152. doi: 10.1016/S0360-1315(99)00029-9 First citation in articleCrossref, Google Scholar

  • Andrich, D. (1978). Application of a psychometric rating model to ordered categories which are scored with successive integers. Applied Psychological Measurement, 2, 581–594. doi: 10.1177/014662167800200413 First citation in articleCrossref, Google Scholar

  • Asseburg, R. & Frey, A. (2013). Too hard, too easy, or just right? The relationship between effort or boredom and ability-difficulty fit. Psychological Test and Assessment Modeling, 55, 92–104. First citation in articleGoogle Scholar

  • Baddeley, A. D. (1986). Working memory. Oxford, UK: Clarendon Press. doi: 10.1002/acp.2350020209 First citation in articleCrossref, Google Scholar

  • Eitel, A., Scheiter, K., Schüler, A., Nyström, M. & Holmqvist, K. (2013). How a picture facilitates the process of learning from text: Evidence for scaffolding. Learning and Instruction, 28, 48–63. doi: 10.1016/j.learninstruc.2013.05.002 First citation in articleCrossref, Google Scholar

  • Fischer, G. H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359–374. doi: 10.1016/0001-6918(73)90003-6 First citation in articleCrossref, Google Scholar

  • Haladyna, T. M., Downing, S. M. & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15, 309–344. doi: 10.1207/S15324818AME1503_5 First citation in articleCrossref, Google Scholar

  • Haladyna, T. M. & Rodriguez, M. C. (2013). Developing and validating test items. New York, NY: Routledge. First citation in articleCrossref, Google Scholar

  • Hartmann, S. (2012). Die Rolle von Leseverständnis und Lesegeschwindigkeit beim Zustandekommen der Leistungen in schriftlichen Tests zur Erfassung naturwissenschaftlicher Kompetenz [The role of reading comprehension and reading speed in text-based assessments of scientific inquiry skills]. Doctoral dissertation, University of Duisburg-Essen. Retrieved from http://duepublico.uni-duisburg-essen.de/servlets/DerivateServlet/Derivate-33260/hartmann_diss.pdf First citation in articleGoogle Scholar

  • Heller, K. A. & Perleth, C. (2000). KFT 4–12+ R: Kognitiver Fähigkeitstest für 4. bis 12. Klassen, Revision [Cognitive Abilities Test for students from grade 4 to 12+ (CogAT; Thorndike, L. & Hagen, E., 1954-1986) German adapted version/author]. Göttingen, Germany: Beltz. First citation in articleGoogle Scholar

  • IEA [International Association for the Evaluation of Educational Achievement]. (2013). TIMSS 2011 assessment released science items Retrieved from http://nces.ed.gov/timss/pdf/TIMSS2011_G4_Science.pdf Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College First citation in articleGoogle Scholar

  • Kunter, M., Schümer, G., Artelt, C., Baumert, J., Klieme, E., Neubrand, M., … Weib, M. (2002). German scale handbook for PISA 2000. Berlin, Germany: Max-Planck-Institut für Bildungsforschung. First citation in articleGoogle Scholar

  • Lenzner, A., Schnotz, W. & Müller, A. (2013). The role of decorative pictures in learning. Instructional Science, 41, 811–831. doi: 10.1007/s11251-012-9256-z First citation in articleCrossref, Google Scholar

  • Levie, H. W. & Lentz, R. (1982). Effects of text illustration: A review of research. Educational Communication & Technology Journal, 30, 195–232. doi: 10.1007/BF02765184 First citation in articleCrossref, Google Scholar

  • Lindner, M. A., Eitel, A., Thoma, G.-B., Dalehefte, I. M., Ihme, J. M. & Köller, O. (2014). Tracking the decision making process in multiple-choice assessment: Evidence from eye movements. Applied Cognitive Psychology, 28, 738–752. doi: 10.1002/acp.3060 First citation in articleCrossref, Google Scholar

  • Martin M. O.Mullis I. V. S.Eds.. (2012). Methods and procedures in TIMSS and PIRLS 2011. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. First citation in articleGoogle Scholar

  • Mayer R. E. (2005). The Cambridge handbook of multimedia learning. Cambridge, UK: University Press. doi: 10.1017/CBO9780511816819.005 First citation in articleCrossref, Google Scholar

  • Messick, S. (1989). Validity. In R. L. LinnEd., Educational measurement (3rd ed., pp. 13–103). New York, NY: Macmillan. First citation in articleGoogle Scholar

  • Möller, J. & Bonerad, E. M (2007). Fragebogen zur habituellen Lesemotivation. [Habitual reading motivation questionnaire]. Psychologie in Erziehung und Unterricht, 54, 259–267. First citation in articleGoogle Scholar

  • Moreno, R. & Mayer, R. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19, 309–326. doi: 10.1007/s10648-007-9047-2 First citation in articleCrossref, Google Scholar

  • Mullis, I. V., Martin, M. O., Ruddock, G. J., O’Sullivan, C. Y. & Preuschoff, C. (2009). TIMSS 2011 Assessment Frameworks. Amsterdam, The Netherlands: International Association for the Evaluation of Educational Achievement (IEA). First citation in articleGoogle Scholar

  • Muthén, L. K. & Muthén, B. O. (1998–2015). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén. First citation in articleGoogle Scholar

  • OECD. (2009). PISA 2006 Technical Report. Paris, France: OECD Publishing. doi: 10.1787/9789264167872-en First citation in articleCrossref, Google Scholar

  • OECD. (2013). PISA 2012 Assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. Paris, France: OECD Publishing. doi: 10.1787/9789264190511-en First citation in articleCrossref, Google Scholar

  • Paivio, A. (1986). Mental representations: A dual coding approach. New York, NY: Oxford University Press. First citation in articleGoogle Scholar

  • Prenzel, M., Häußler, P., Rost, J. & Senkbeil, M. (2002). Der PISA-Naturwissenschaftstest: Lassen sich die Aufgabenschwierigkeiten vorhersagen? [The PISA science literacy test: Are the item difficulties predictable?]. Unterrichtswissenschaft, 30(2), 120–135. doi: nbn:de:0111-opus-76826 First citation in articleGoogle Scholar

  • Raftery, A. E. (1999). Bayes factors and BIC: Comment on “A critique of the Bayesian information criterion for model selection”. Sociological Methods & Research, 27, 411–417. doi: 10.1177/0049124199027003005 First citation in articleCrossref, Google Scholar

  • Retelsdorf, J., Becker, M., Köller, O. & Möller, J. (2012). Reading development in a tracked school system: A longitudinal study over 3 years using propensity score matching. British Journal of Educational Psychology, 82, 647–671. doi: 10.1111/j.2044-8279.2011.02051.x First citation in articleCrossref, Google Scholar

  • Saß, S., Wittwer, J., Senkbeil, M. & Köller, O. (2012). Pictures in test items: Effects on response time and response correctness. Applied Cognitive Psychology, 26, 70–81. doi: 10.1002/acp.1798 First citation in articleCrossref, Google Scholar

  • Schneider, W., Schlagmüller, M. & Ennemoser, M. (2007). LGVT 6–12: Lesegeschwindigkeits- und -verständnistest für die Klassen 6–12 [LGVT 6–12: A reading comprehension test for students from grade 6 to 12]. Göttingen, Germany: Hogrefe. First citation in articleGoogle Scholar

  • Schnotz, W. & Bannert, M. (2003). Construction and interference in learning from multiple representation. Learning and Instruction, 13, 141–156. doi: 10.1016/S0959-4752(02)00017-8 First citation in articleCrossref, Google Scholar

  • Schnotz, W., Ludewig, U., Ullrich, M., Horz, H., McElvany, N. & Baumert, J. (2014). Strategy shifts during learning from texts and pictures. Journal of Educational Psychology, 106, 974–989. doi: 10.1037/a0037054 First citation in articleCrossref, Google Scholar

  • Steyer, R., Eid, M. & Schwenkmezger, P. (1997). Modeling true intraindividual change: True change as a latent variable. Methods of Psychological Research Online, 2, 21–33. First citation in articleGoogle Scholar

  • Vekiri, I. (2002). What is the value of graphical displays in learning? Educational Psychology Review, 14, 261–312. doi: 10.1023/A:1016064429161 First citation in articleCrossref, Google Scholar

  • Wise, S. L. & DeMars, C. E. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10, 1–17. doi: 10.1207/s15326977ea1001_1 First citation in articleCrossref, Google Scholar

  • Wise, S. L., Pastor, D. A. & Kong, X. J. (2009). Correlates of rapid-guessing behavior in low-stakes testing: Implications for test development and measurement practice. Applied Measurement in Education, 22, 185–205. doi: 10.1080/08957340902754650 First citation in articleCrossref, Google Scholar

  • Wu, H. K., Kuo, C. Y., Jen, T. H. & Hsu, Y. S. (2015). What makes an item more difficult? Effects of modality and type of visual information in a computer-based assessment of scientific inquiry abilities. Computers & Education, 85, 35–48. doi: 10.1016/j.compedu.2015.01.007 First citation in articleCrossref, Google Scholar