Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

I Will Probably Fail

Higher Ability Students’ Motivational Experiences During Adaptive Achievement Testing

Published Online:January 01, 2014https://doi.org/10.1027/1015-5759/a000168

Abstract

We investigated the effects of computerized adaptive testing (CAT) versus computerized fixed item testing (FIT) of reasoning ability on current motivation in terms of situational fear of failure and subjective probability of success, as well as flow. A group of 174 students (aged 15–21) from two German secondary schools was presented either a CAT or a FIT version of a matrices test; motivational variables were assessed during a short break in testing. More situational fear of failure and less subjective probability of success were reported using CAT compared to FIT. Self-reported flow did not differ between test mode conditions. When we addressed the hypothesis that adaptive testing is equally motivating for both high and lower performers, test performance appeared to moderate the relationship of test mode and subjective probability of success: Only during FIT was subjective probability of success higher with lower test performance. This moderation effect was also revealed for the relationship of test mode and flow. However, as average reported motivation was lower during CAT, results contradict assumptions of enhanced motivation during CAT. Results are discussed in relation to self-concept relevance of testing domains and with reference to test fairness.

References

  • Aiken, L. S. , West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Thousand Oaks, CA: Sage. First citation in articleGoogle Scholar

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education . (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. First citation in articleGoogle Scholar

  • Arvey, R. D. , Strickland, W. , Drauden, G. , Martin, C. (1990). Motivational components of test taking. Personnel Psychology, 43, 695–716. doi 10.1111/j.1744-6570.1990.tb00679.x First citation in articleCrossref, Google Scholar

  • Betz, N. E. , Weiss, D. J. (1976). Psychological effects of immediate knowledge of results and adaptive ability testing (Research Report 76–4). Minneapolis, MN: University of Minnesota, Department of Psychology. First citation in articleGoogle Scholar

  • Cassady, J. C. , & Johnson, R. E. (2002). Cognitive test anxiety and academic performance. Contemporary Educational Psychology, 27, 270–295. doi 10.1006/ceps.2001.1094 First citation in articleCrossref, Google Scholar

  • Csikszentmihalyi, M. (2000). Beyond boredom and anxiety. San Francisco, CA: Jossey-Bass. First citation in articleGoogle Scholar

  • Dauenheimer, D. , Stahlberg, D. , Spreemann, S. , Sedikides, C. (2002). Self-enhancement, self-verification, or self-assessment? The intricate role of trait modifiability in the self-evaluation process. Revue Internationale de Psychologie Sociale, 15, 89–112. First citation in articleGoogle Scholar

  • Eccles, J. S. , Adler, T. F. , Futterman, R. , Goff, S. B. , Kaczala, C. M. , Meece, J. L. , Midgley, C. (1983). Expectancies, values, and academic behaviors. In J. T. Spence, (Ed.), Achievement and achievement motives (pp. 75–146). San Francisco, CA: Freeman. First citation in articleGoogle Scholar

  • Eggen, T. J. H. M. , & Verschoor, A. J. (2006). Optimal testing with easy or difficult items in computerized adaptive testing. Applied Psychological Measurement, 30, 379–393. doi 10.1177/ 0146621606288890 First citation in articleCrossref, Google Scholar

  • Engeser, S. , Rheinberg, F. (2008). Flow, performance and moderators of challenge-skill balance. Motivation and Emotion, 32, 158–172. doi 10.1007/s11031-008-9102-4 First citation in articleCrossref, Google Scholar

  • Frey, A. , Hartig, J. , & Moosbrugger, H. (2009). Effects of adaptive testing on testtaking motivation with the example of the Frankfurt Adaptive Concentration Test. Diagnostica, 55, 20–28. doi 10.1026/0012-1924.55.1.20 First citation in articleLink, Google Scholar

  • Green, J. D. , Pinter, B. , & Sedikides, C. (2005). Mnemic neglect and self-threat: Trait modifiability moderates self-protection. European Journal of Social Psychology, 35, 225–235. doi 10.1002/ejsp.242 First citation in articleCrossref, Google Scholar

  • Hagtvet, K. A. , & Benson, J. (1997). The motive to avoid failure and test anxiety responses: Empirical support for integration of two research traditions. Anxiety, Stress, and Coping, 10, 35–57. doi 10.1080/10615809708249294 First citation in articleCrossref, Google Scholar

  • Häusler, J. , & Sommer, M. (2008). The effect of success probability on test economy and self-confidence in computerized adaptive tests. Psychology Science, 50, 75–87. First citation in articleGoogle Scholar

  • Helms, J. E. (2006). Fairness is not validity or cultural bias in racial group assessment: A quantitative perspective. American Psychologist, 61, 845–859. doi 10.1037/0003-066X.61.8.845 First citation in articleCrossref, Google Scholar

  • Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58, 47–77. doi 10.3102/00346543058001047 First citation in articleCrossref, Google Scholar

  • Hornke, L. F. , Etzel, S. , Rettig, K. , & Prieler, J. (2005). Adaptive Matrices Test. Mödling: Schuhfried. First citation in articleGoogle Scholar

  • Hornke, L. F. , Küppers, A. , Etzel, S. (2000). Design and evaluation of an adaptive matrices test. Diagnostica, 46, 182–188. doi 10.1026//0012-1924.46.4.182 First citation in articleLink, Google Scholar

  • Keller, J. , Bless, H. (2008). Flow and regulatory compatibility: An experimental approach to the flow model of intrinsic motivation. Personality and Social Psychology Bulletin, 34, 196–209. doi 10.1177/0146167207310026 First citation in articleCrossref, Google Scholar

  • Linacre, J. M. (2000). Computer-adaptive testing: A methodology whose time has come. Memo 69. Retrieved from www.rasch.org/memos.htm First citation in articleGoogle Scholar

  • Meece, J. L. , Bower Glienke, B. , Burg, S. (2006). Gender and motivation. Journal of School Psychology, 44, 351–373. doi 10.1016/j.jsp.2006.04.004 First citation in articleCrossref, Google Scholar

  • Ortner, T. M. , & Caspers, J. (2011). Consequences of test anxiety on adaptive versus fixed item testing. European Journal of Psychological Assessment, 27, 157–163. doi 10.1027/1015-5759/a000062 First citation in articleLink, Google Scholar

  • Ortner, T. M. , Weißkopf, E. , Gerstenberg, F. R. (2012). Skilled but unaware of it: CAT undermines a testtaker’s metacognitive competence. European Journal of Psychology of Education, 28, 1–15. doi 10.1007/s10212-011-0100-7 First citation in articleGoogle Scholar

  • Pine, S. M. , Church, A. T. , Gialluca, K. A. , Weiss, D. J. (1979). Effects of computerized adaptive testing on black and white students (Research report 79–2). Minneapolis, MN: University of Minnesota, Department of Psychology. First citation in articleGoogle Scholar

  • Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Nielsen & Lydiche. First citation in articleGoogle Scholar

  • Reschly, D. J. , & Robinson-Zanartu, C. (2000). Evaluation of aptitudes. In G. Goldstein, M. Hersen, (Eds.), Handbook of psychological assessment (pp. 183–202). Oxford, UK: Elsevier. First citation in articleCrossref, Google Scholar

  • Rheinberg, F. , Vollmeyer, R. , Burns, B. D. (2001). FAM: Ein Fragebogen zur Erfassung aktueller Motivation in Lern- und Leistungssituationen [QCM: A questionnaire to assess motivation in learning situations]. Diagnostica, 47, 57–66. doi 10.1026//0012-1924.47.2.57 First citation in articleLink, Google Scholar

  • Rheinberg, F. , Vollmeyer, R. , Engeser, S. (2003). Die Erfassung des Flow-Erlebens [The assessment of flow experience]. In J. Stiensmeier-Pelster, F. Rheinberg, (Eds.), Diagnostik von Selbstkonzept, Lernmotivation und Selbstregulation (pp. 261– 279). Göttingen: Hogrefe. First citation in articleGoogle Scholar

  • Robbins, S. B. , Lauver, K. , Le, H. , Davis, D. , Langley, R. , Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261–288. doi 10.1037/0033-2909.130.2.261 First citation in articleCrossref, Google Scholar

  • Seipp, B. (1991). Anxiety and academic performance: A meta-analysis of findings. Anxiety Research, 4, 27–41. First citation in articleCrossref, Google Scholar

  • Steinmayr, R. , & Spinath, B. (2008). Sex differences in school achievement: What are the roles of personality and achievement motivation? European Journal of Personality, 22, 185–209. doi 10.1002/per.676 First citation in articleCrossref, Google Scholar

  • Thissen, D. , Mislevy, R. J. (2000). Computerized adaptive testing: A primer. In H. Wainer, (Ed.), Testing algorithms (pp. 475–496). Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Tonidandel, S. , & Quinones, M. A. (2000). Psychological reactions to adaptive testing. International Journal of Selection and Assessment, 8, 7–15. doi 10.1111/1468-2389.00126 First citation in articleCrossref, Google Scholar

  • Tonidandel, S. , Quinones, M. A. , Adams, A. A. (2002). Computer-adaptive testing: The impact of test characteristics on perceived performance and testtakers’ reactions. Journal of Applied Psychology, 87, 320–332. doi 10.1037//0021-9010.87.2.320 First citation in articleCrossref, Google Scholar

  • Truxillo, D. M. , Bodner, T. E. , Bertolino, M. , Bauer, T. N. , Yonce, C. A. (2009). Effects of explanations on applicant reactions: A meta-analytic review. International Journal of Selection and Assessment, 17, 346–360. doi 10.1111/j.1468-2389.2009.00478.x First citation in articleCrossref, Google Scholar

  • Van der Linden, W. J. , Glas, C. A. W. (Eds.). (2000). Computerized adaptive testing: Theory and practice. St. Paul, MN: Assessment Systems Corporation. First citation in articleCrossref, Google Scholar

  • Vollmeyer, R. , & Rheinberg, F. (1998). Motivationale Einflüsse auf Erwerb und Anwendung von Wissen in einem computersimulierten System [Motivational influences on the acquisition and application of knowledge in a computer-simulated system]. Zeitschrift für Pädagogische Psychologie, 12, 11–23. First citation in articleGoogle Scholar

  • Vollmeyer, R. , & Rheinberg, F. (2006). Motivational effects on self-regulated learning with different tasks. Educational Psychology Review, 18, 239–253. doi 10.1007/s10648-006-9017-0 First citation in articleCrossref, Google Scholar

  • Wainer, H. , Dorans, N. J. , Flaugher, R. , Green, B. F. , Mislevy, R. J. , Steinberg, L. , Thissen, D. (1990). Computerized adaptive testing: A primer. Hillsdale, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Wainer, H. , Eignor, D. (2000). Caveats, pitfalls, and unexpected consequences of implementing large scale computerized testing. In H. Wainer, (Ed.), Computerized adaptive testing: A primer (pp. 271–299). Hillsdale, NJ: Erlbaum. First citation in articleCrossref, Google Scholar

  • Weiss, D. J. , Betz, N. E. (1973). Ability measurement: conventional or adaptive? (Research report 73–1). Minneapolis, MN: University of Minnesota, Department of Psychology. First citation in articleGoogle Scholar

  • Wigfield, A. , Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25, 68–81. doi 10.1006/ceps.1999.1015 First citation in articleCrossref, Google Scholar