Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Incremental Validity of Online Over Offline Reports of Volitional Control in Predicting Learning Success

Published Online:May 26, 2021https://doi.org/10.1026/0049-8637/a000219

Abstract

Abstract. Volitional control (i. e., efforts to maintain goal striving in the face of obstacles) is an integral part of self-regulated learning and an important factor for explaining individual differences in academic performance. However, differences between the various methods for assessing volitional control have rarely been investigated. Two common methods are (a) offline questionnaires, in which respondents aggregate experiences over a longer period of time, and (b) online questionnaires such as learning diaries, which assess respondents’ experiences close to the learning event. We compared these assessment approaches in 96 medical students who prepared for a high-stakes exam. Achievement of self-set learning goals was measured objectively via logfiles of students’ activities on a learning platform. Daily reports of volitional control explained substantial variance in the achievement of learning goals over and above the offline questionnaire, indicating incremental validity of online assessments of self-regulation. Moreover, the daily reports of volitional control could explain intra-individual day-to-day variance in goal achievement. The current study, thus, suggests that learning diaries, albeit cumbersome, have clear advantages over offline questionnaires.


Inkrementelle Validität einer täglichen gegenüber einer retrospektiven Erfassung von volitionaler Kontrolle für die Vorhersage von Lernerfolg

Zusammenfassung. Volitionale Kontrolle ist ein integraler Bestandteil selbstregulierten Lernens und ein wichtiger Faktor zur Erklärung individueller Unterschiede in der akademischen Leistung. Unterschiedliche Erhebungsmethoden zur Erfassung volitionaler Kontrolle wurden bisher jedoch selten systematisch untersucht. Zwei gebräuchliche Methoden sind a) Offline-Fragebogen, in welchen über einen längeren Zeitraum aggregierte Lernerfahrungen erfragt werden und b) Online-Fragebogen wie etwa Lerntagebücher, die Lernerfahrungen zeitnah erfassen. Wir verglichen die beiden Erhebungsmethoden bei 96 Medizinstudierenden, die sich auf das Zweite Staatsexamen vorbereiteten. Das Erreichen selbstgesetzter Lernziele wurde mittels Logfile-Daten objektiv gemessen. Die täglichen Einschätzungen der Studierenden erklärten substantielle Varianz in der Lernzielerreichung über den Offline-Fragebogen hinaus, was für die inkrementelle Validität von Online-Erhebungen der Selbstregulation spricht. Darüber hinaus erklärten die Lerntagebuchdaten zur volitionalen Kontrolle intra-individuelle tägliche Schwankungen in der Lernzielerreichung. Die vorliegende Studie legt daher nahe, dass Lerntagebucherfassungen, obgleich aufwändig, deutliche Vorteile gegenüber Offline-Fragebogen haben.

Literatur

  • Boekaerts, M., & Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment and intervention. Applied Psychology, 54 (2), 199 – 231. https://doi.org/10.1111/j.1464-0597.2005.00205.x First citation in articleCrossref, Google Scholar

  • Bolger, N., & Laurenceau, J.-P. (2013). Intensive longitudinal methods. New York, NY: The Guilford Press. First citation in articleGoogle Scholar

  • Breitwieser, J., Neubauer, A. B., Schmiedek, F., & Brod, G. (2020). Self-regulation prompts promote the achievement of learning goals – but only briefly: Uncovering hidden dynamics in the effects of a psychological intervention. https://doi.org/10.31234/osf.io/bz49 m First citation in articleGoogle Scholar

  • Conner, T. S., & Barrett, L. F. (2012). Trends in ambulatory self-report: The role of momentary experience in psychosomatic medicine. Psychosomatic Medicine, 74 (4), 327 – 337. https://doi.org/10.1097/PSY.0b013e3182546f18 First citation in articleCrossref, Google Scholar

  • Conner, T. S., & Mehl, M. R. (2015). Ambulatory assessment: Methods for studying everyday life. In R. ScottS. KosslynN. Pinkerton (Eds.), Emerging trends in the social and behavioral sciences (pp. 1 – 15). Hoboken, NJ: Wiley. https://doi.org/10.1002/9781118900772.etrds0010 First citation in articleCrossref, Google Scholar

  • Corno, L. (1993). The best-laid plans: Modern conceptions of volition and educational research. Educational Researcher, 22 (2), 14 – 22. https://doi.org/10.3102/0013189X022002014 First citation in articleCrossref, Google Scholar

  • Corno, L. (1994). Student volition and education: Outcomes, influences, and practices. In D. H. SchunkB. J. Zimmerman (Eds.), Self-regulation of learning and performance: issues and educational applications (pp. 229 – 254). Lawrence Erlbaum Associates. First citation in articleGoogle Scholar

  • Corno, L., & Kanfer, R. (1993). The role of volition in learning and performance. Review of Research in Education, 19 (1), 301 – 341. https://doi.org/10.3102/0091732X019001301 First citation in articleCrossref, Google Scholar

  • Dent, A. L., & Koenka, A. C. (2016). The relation between self-regulated learning and academic achievement across childhood and adolescence: A meta-analysis. Educational Psychology Review, 28 (3), 425 – 474. https://doi.org/10.1007/s10648-015-9320-8 First citation in articleCrossref, Google Scholar

  • Dörrenbächer, L., & Perels, F. (2015). Volition completes the puzzle: Development and evaluation of an integrative trait model of self-regulated learning. Frontline Learning Research, 3 (4), 14 – 36. https://doi.org/10.14786/flr.v3i4.179 First citation in articleGoogle Scholar

  • Dörrenbächer, L., & Perels, F. (2016). More is more? Evaluation of interventions to foster self-regulated learning in college. International Journal of Educational Research, 78, 50 – 65. https://doi.org/10.1016/j.ijer.2016.05.010 First citation in articleCrossref, Google Scholar

  • Duckworth, A. L., & Kern, M. (2011). A meta-analysis of convergent validity evidence for self-control measures. Journal of Personality Research, 45 (3), 259 – 268. https://doi.org/10.1016/j.jrp.2011.02.004 First citation in articleCrossref, Google Scholar

  • Duckworth, A. L., Taxer, J. L., Eskreis-Winkler, L., Galla, B. M., & Gross, J. J. (2019). Self-control and academic achievement. Annual Review of Psychology, 70 (1), 373 – 399. https://doi.org/10.1146/annurev-psych-010418-103230 First citation in articleCrossref, Google Scholar

  • Klug, J., Ogrin, S., Keller, S., Ihringer, A., & Schmitz, B. (2011). A plea for self-regulated learning as a process: Modelling, measuring and intervening. Psychological Test and Assessment Modeling, 53 (1), 51 – 72. First citation in articleGoogle Scholar

  • Korotitsch, W. J., & Nelson-Gray, R. O. (1999). An overview of self-monitoring research in assessment and treatment. Psychological Assessment, 11 (4), 415 – 425. https://doi.org/10.1037/1040-3590.11.4.415 First citation in articleCrossref, Google Scholar

  • Kuhl, J. (1985). Volitional mediators of cognition-behavior consistency: Self-regulatory processes and action versus state orientation. In J. KuhlJ. Beckmann (Eds.), Action control (pp. 101 – 128). Berlin: Springer. https://doi.org/10.1007/978-3-642-69746-3_6 First citation in articleCrossref, Google Scholar

  • Kuhl, J. (2000). A functional-design approach to motivation and self-regulation: The dynamics of personality systems interactions. In M. BoekaertsP. R. PintrichM. Zeidner (Eds.), Handbook of self-regulation (pp. 111 – 169). Academic Press. https://doi.org/10.1016/B978-012109890-2/50034-2 First citation in articleCrossref, Google Scholar

  • Liborius, P., Bellhäuser, H., & Schmitz, B. (2019). What makes a good study day? An intraindividual study on university students’ time investment by means of time-series analyses. Learning and Instruction, 60, 310 – 321. https://doi.org/10.1016/j.learninstruc.2017.10.006 First citation in articleCrossref, Google Scholar

  • McCann, E. J., & Garcia, T. (1999). Maintaining motivation and regulating emotion: Measuring individual differences in academic volitional strategies. Learning and Individual Differences, 11 (3), 259 – 279. https://doi.org/10.1016/S1041-6080(99)80003-X First citation in articleCrossref, Google Scholar

  • Mouratidis, A., Vansteenkiste, M., Michou, A., & Lens, W. (2013). Perceived structure and achievement goals as predictors of students’ self-regulated learning and affect and the mediating role of competence need satisfaction. Learning and Individual Differences, 23 (1), 179 – 186. https://doi.org/10.1016/j.lindif.2012.09.001 First citation in articleCrossref, Google Scholar

  • Muthén, L. K., & Muthén, B. O. (2017). Mplus User’s Guide (8th ed.). Los Angeles, CA: Muthén & Muthén. First citation in articleGoogle Scholar

  • O’Brien, R. M. (2007). A caution regarding rules of thumb for variance inflation factors. Quality and Quantity, 41 (5), 673 – 690. https://doi.org/10.1007/s11135-006-9018-6 First citation in articleCrossref, Google Scholar

  • Panadero, E., Klug, J., & Järvelä, S. (2016). Third wave of measurement in the self-regulated learning field: When measurement and intervention come hand in hand. Scandinavian Journal of Educational Research, 60 (6), 723 – 735. https://doi.org/10.1080/00313831.2015.1066436 First citation in articleCrossref, Google Scholar

  • Perels, F., Otto, B., Landmann, M., Hertel, S., & Schmitz, B. (2007). Self-regulation from a process perspective. Zeitschrift Für Psychologie / Journal of Psychology, 215 (3), 194 – 204. https://doi.org/10.1027/0044-3409.215.3.194 First citation in articleLink, Google Scholar

  • Pintrich, P. R. (2004). A conceptual framework for assessing motivation and SRL in college students. Educational Psychology Review, 16 (4), 385 – 407. First citation in articleCrossref, Google Scholar

  • Pintrich, P. R., Wolters, C. a, & Baxter, G. P. (2000). Assessing metacognition and self-regulated learning. Issues in the Measurement of Metacognition, 43 – 97. Available at http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1002&context=burosmetacognition First citation in articleGoogle Scholar

  • Rausch, A. (2014). Using diaries in research on work and learning. In C. HarteisA. RauschJ. Seifried (Eds.), Discourses on professional learning: On the boundary between learning and working (pp. 341 – 366). Berlin: Springer. https://doi.org/10.1007/978-94-007-7012-6_17 First citation in articleCrossref, Google Scholar

  • Rhemtulla, M., van Bork, R., & Borsboom, D. (2020). Worse than measurement error: Consequences of inappropriate latent variable measurement models. Psychological Methods, 25 (1), 30 – 45. https://doi.org/10.1037/met0000220 First citation in articleCrossref, Google Scholar

  • Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138 (2), 353 – 387. https://doi.org/10.1037/a0026838 First citation in articleCrossref, Google Scholar

  • Robinson, M. D., & Clore, G. L. (2002). Belief and feeling: Evidence for an accessibility model of emotional self-report. Psychological Bulletin, 128 (6), 934 – 960. https://doi.org/10.1037/0033-2909.128.6.934 First citation in articleCrossref, Google Scholar

  • Rovers, S. F. E., Clarebout, G., Savelberg, H. H. C. M., de Bruin, A. B. H., & van Merriënboer, J. J. G. (2019). Granularity matters: Comparing different ways of measuring self-regulated learning. Metacognition and Learning, 14 (1) https://doi.org/10.1007/s11409-019-09188-6 First citation in articleCrossref, Google Scholar

  • Saunders, B., Milyavskaya, M., Etz, A., Randles, D., & Inzlicht, M. (2018). Reported self-control is not meaningfully associated with inhibition-related executive function: A Bayesian analysis. Collabra: Psychology, 4 (1), 39 https://doi.org/10.1525/collabra.134 First citation in articleCrossref, Google Scholar

  • Schmitz, B., & Perels, F. (2011). Self-monitoring of self-regulation during math homework behaviour using standardized diaries. Metacognition and Learning, 6 (3), 255 – 273. https://doi.org/10.1007/s11409-011-9076-6 First citation in articleCrossref, Google Scholar

  • Schmitz, B., & Skinner, E. (1993). Perceived control, effort, and academic performance: Interindividual, intraindividual, and multivariate time-series analyses. Journal of Personality and Social Psychology, 64 (6), 1010 – 1028. https://doi.org/10.1037/0022-3514.64.6.1010 First citation in articleCrossref, Google Scholar

  • Schmitz, B., & Wiese, B. S. (2006). New perspectives for the evaluation of training sessions in self-regulated learning: Time-series analyses of diary data. Contemporary Educational Psychology, 31 (1), 64 – 96. https://doi.org/10.1016/j.cedpsych.2005.02.002 First citation in articleCrossref, Google Scholar

  • Schwarzer, R., & Jerusalem, M. (1999). Skalen zur Erfassung von Lehrer- und Schülermerkmalen. In R. Schwarzer (Ed.), , Dokumentation der psychometrischen Verfahren im Rahmen der Wissenschaftlichen Begleitung des Modellversuchs Selbstwirksame Schulen. Berlin: Freie Universität. First citation in articleGoogle Scholar

  • Shing, Y. L., & Brod, G. (2016). Effects of prior knowledge on memory: Implications for education. Mind, Brain, and Education, 10 (3), 153 – 161. https://doi.org/10.1111/mbe.12110 First citation in articleCrossref, Google Scholar

  • Tourangeau, R. (2000). Remembering what happened: Memory errors and survey reports. In A. A. StoneJ. S. TurkkanC. A. BachrachJ. B. JobeH. S. KurtzmanV. S. Cain (Eds.), The science of self-report: Implications for research and practice (pp. 29 – 47). Hillsdal, NJ: Lawrence Erlbaum. First citation in articleGoogle Scholar

  • Trull, T. J., & Ebner-Priemer, U. (2013). Ambulatory assessment. Annual Review of Clinical Psychology, 9 (1), 151 – 176. https://doi.org/10.1146/annurev-clinpsy-050212-185510 First citation in articleCrossref, Google Scholar

  • Veenman, M. V. J. (2011). Alternative assessment of strategy use with self-report instruments: A discussion. Metacognition and Learning, 6 (2), 205 – 211. https://doi.org/10.1007/s11409-011-9080-x First citation in articleCrossref, Google Scholar

  • Wallin, P., & Adawi, T. (2018). The reflective diary as a method for the formative assessment of self-regulated learning. European Journal of Engineering Education, 43 (4), 507 – 521. https://doi.org/10.1080/03043797.2017.1290585 First citation in articleCrossref, Google Scholar

  • Wennerhold, L., & Friese, M. (2020). Why self-report measures of self-control and inhibition tasks do not substantially correlate. Collabra: Psychology, 6 (1), 9 https://doi.org/10.1525/collabra.276 First citation in articleCrossref, Google Scholar

  • Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In M. BoekaertsP. R. PintrichM. Zeidner (Eds.), Handbook of self-regulation (pp. 531 – 566). Cambridge, MA: Academic Press. https://doi.org/10.1016/B978-012109890-2/50045-7 First citation in articleCrossref, Google Scholar

  • Wolters, C. A. (1999). The relation between high school students’ motivational regulation and their use of learning strategies, effort, and classroom performance. Learning and Individual Differences, 11 (3), 281 – 299. https://doi.org/10.1016/S1041-6080(99)80004-1 First citation in articleCrossref, Google Scholar

  • Wolters, C. A. (2003). Regulation of motivation: Evaluating an underemphasized aspect of self-regulated learning. Educational Psychologist, 38 (4), 189 – 205. https://doi.org/10.1207/S15326985EP3804_1 First citation in articleCrossref, Google Scholar

  • Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. BoekaertsP. R. PintrichM. Zeidner (Eds.), Handbook of self-regulation (pp. 13 – 39). Cambridge, MA: Academic Press. First citation in articleGoogle Scholar