Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

VOTAT in Action

Exploring Epistemic Activities in Knowledge-Lean Problem-Solving Processes

Published Online:April 24, 2024https://doi.org/10.1027/2151-2604/a000559

Abstract

Abstract: In our study, we integrated the vary-one-thing-at-a-time (VOTAT) strategy within epistemic activities in knowledge-lean problem-solving. VOTAT, altering a single variable while keeping others constant, was examined through a dual approach: an empirical analysis with MicroDYN tasks and a theoretical discussion. Empirically, we identified differences among participants who fully engaged in the VOTAT loop, those with incomplete loops, and nonusers. These distinctions were evident in their problem-solving approaches, hypothesis formulation, and decision-making efficiency. Theoretically, we contextualized VOTAT's role within epistemic activities, extending its application beyond the specific tasks in our study. Our findings reveal that VOTAT's use markedly impacts problem-solving processes, demonstrating varied cognitive strategies among different participant groups. This has implications for understanding problem-solving cognition and designing tasks that better capture the nuances of epistemic activities.

References

  • Baker, R. S. (2019). Challenges for the future of educational data mining: The baker learning analytics prizes. Journal of Educational Data Mining, 11(1), 1–17. 10.5281/ZENODO.3554745 First citation in articleCrossref, Google Scholar

  • Brandl, L., Richters, C., Radkowitsch, A., Obersteiner, A., & Stadler, M. (2021). Simulation-based learning of complex skills: Predicting performance with theoretically derived process features. Psychological Test and Assessment Modeling, 63(4), 542–560. First citation in articleGoogle Scholar

  • Bunderson, C. V., Inouye, D. K., & Olsen, J. B. (1988). The four generations of computerized educational measurement. ETS Research Report Series, 1988(1), i–148. 10.1002/j.2330-8516.1988.tb00291.x First citation in articleCrossref, Google Scholar

  • Dörner, D., & Funke, J. (2017). Complex problem solving: What it is and what it is not. Frontiers in Psychology, 8, Article 1153. 10.3389/fpsyg.2017.01153 First citation in articleCrossref, Google Scholar

  • Fischer, F., Kollar I., Ufer, S., Sodian, B., Hussmann, H., Pekrun, R., Neuhaus, B., Dorner, B., Pankofer, S., Fischer, M., Strijbos, J.-W., Heene, M., & Eberle, J. (2014). Scientific reasoning and argumentation: Advancing an interdisciplinary research agenda in education. Frontline Learning Research, 2(2), 28–45. 10.14786/flr.v2i2.96 First citation in articleCrossref, Google Scholar

  • Funke, J. (2001). Dynamic systems as tools for analysing human judgement. Thinking and Reasoning, 7(1), 69–89. 10.1080/13546780042000046 First citation in articleCrossref, Google Scholar

  • Gabadinho, A., Ritschard, G., Studer, M., & Müller, N. S. (2009). Mining sequence data in R with the TraMineR package: A user's guide. http://mephisto.unige.ch/pub/traminer/doc/1.8/traminer-1.8-users-guide.pdf First citation in articleGoogle Scholar

  • Giannakos, M., & Cukurova, M. (2023). The role of learning theory in multimodal learning analytics. British Journal of Educational Technology, 54(5), 1246–1267. 10.1111/bjet.13320 First citation in articleCrossref, Google Scholar

  • Goldhammer, F., Hahnel, C., Kroehne, U., & Zehner, F. (2021). From byproduct to design factor: On validating the interpretation of process indicators based on log data. Large-Scale Assessments in Education, 9(1), Article 20. 10.1186/s40536-021-00113-5 First citation in articleCrossref, Google Scholar

  • Goldhammer, F., Naumann, J., Rölke, H., Stelter, A., & Tóth, K. (2017). Relating product data to process data from computer-based competency assessment. In D. LeutnerJ. FleischerJ. GrünkornE. Klieme (Eds.), Competence assessment in education (pp. 407–425). Springer. 10.1007/978-3-319-50030-0_24 First citation in articleCrossref, Google Scholar

  • Greiff, S., Fischer, A., Stadler, M., & Wüstenberg, S. (2015). Assessing complex problem-solving skills with multiple complex systems. Thinking & Reasoning, 21(3), 356–382. 10.1080/13546783.2014.989263 First citation in articleCrossref, Google Scholar

  • Greiff, S., Niepel, C., Scherer, R., & Martin, R. (2016). Understanding students' performance in a computer-based assessment of complex problem solving: An analysis of behavioral data from computer-generated log files. Computers in Human Behavior, 61, 36–46. 10.1016/j.chb.2016.02.095 First citation in articleCrossref, Google Scholar

  • Greiff, S., Wüstenberg, S., & Avvisati, F. (2015). Computer-generated log-file analyses as a window into students' minds? A showcase study based on the PISA 2012 assessment of problem solving. Computers & Education, 91, 92–105. 10.1016/j.compedu.2015.10.018 First citation in articleCrossref, Google Scholar

  • Hahnel, C., Jung, A. J., & Goldhammer, F. (2023). Theory matters. European Journal of Psychological Assessment, 39(4), 271–279. 10.1027/1015-5759/a000776 First citation in articleLink, Google Scholar

  • Hahnel, C., Ramalingam, D., Kroehne, U., & Goldhammer, F. (2023). Patterns of reading behaviour in digital hypertext environments. Journal of Computer Assisted Learning, 39(3), 737–750. 10.1111/jcal.12709 First citation in articleCrossref, Google Scholar

  • He, Q., Borgonovi, F., & Paccagnella, M. (2019). Using process data to understand adults’ problem-solving behaviour in the programme for the international assessment of adult competencies (PIAAC): Identifying generalised patterns across multiple tasks with sequence mining. 10.1787/650918f2-en First citation in articleCrossref, Google Scholar

  • He, Q., Borgonovi, F., & Paccagnella, M. (2021). Leveraging process data to assess adults’ problem-solving skills: Using sequence mining to identify behavioral patterns across digital tasks. Computers & Education, 166, Article 104170. 10.1016/j.compedu.2021.104170 First citation in articleCrossref, Google Scholar

  • He, Q., & von Davier, M. (2015). Identifying feature sequences from process data in problem-solving items with N-grams. In L. A. van der ArkD. M. BoltW.-C. WangJ. A. DouglasS.-M. Chow (Eds.), Springer proceedings in mathematics & statistics. Quantitative psychology research (Vol. 140, pp. 173–190). Springer International Publishing. 10.1007/978-3-319-19977-1_13 First citation in articleCrossref, Google Scholar

  • Helske, S., & Helske, J. (2019). Mixture hidden Markov models for sequence data: The seqHMM Package in R. Journal of Statistical Software, 88(3), 1–32. 10.18637/jss.v088.i03 First citation in articleCrossref, Google Scholar

  • Knight, S., & Buckingham Shum, S. (2017). Theory and learning analytics. In C. LangG. SiemensA. WiseD. Gašević (Eds.), Handbook of learning analytics (pp. 17–22). Society for Learning Analytics Research (SoLAR). 10.18608/hla17.001 First citation in articleCrossref, Google Scholar

  • Krieger, F., Stadler, M., Bühner, M., Fischer, F., & Greiff, S. (2021). Assessing complex problem-solving skills in under 20 minutes. Psychological Test Adaptation and Development, 2(1), 80–92. 10.1027/2698-1866/a000009 First citation in articleLink, Google Scholar

  • Kröner, S., Plass, J., & Leutner, D. (2005). Intelligence assessment with computer simulations. Intelligence, 33(4), 347–368. 10.1016/j.intell.2005.03.002 First citation in articleCrossref, Google Scholar

  • Kuhn, D., & Dean, D. (2005). Is developing scientific thinking all about learning to control variables? Psychological Science, 16(11), 866–870. 10.1111/j.1467-9280.2005.01628.x First citation in articleCrossref, Google Scholar

  • Lotz, C., Scherer, R., Greiff, S., & Sparfeldt, J. R. (2017). Intelligence in action – effective strategic behaviors while solving complex problems. Intelligence, 64, 98–112. 10.1016/j.intell.2017.08.002 First citation in articleCrossref, Google Scholar

  • Lotz, C., Scherer, R., Greiff, S., & Sparfeldt, J. R. (2022). g's little helpers – VOTAT and NOTAT mediate the relation between intelligence and complex problem solving. Intelligence, 95, Article 101685. 10.1016/j.intell.2022.101685 First citation in articleCrossref, Google Scholar

  • OECD (2014). Creative problem solving: Students' skills in tackling real-life problems. PISA 2012 results/Organisation for Economic Co-operation and Development, Programme for International Student Assessment: Vol. V. https://www.voced.edu.au/content/ngv:63012 First citation in articleGoogle Scholar

  • R Core Team (2022). R: A language and environment for statistical computing [Computer software]. Statistical Computing. https://www.R-project.org/ First citation in articleGoogle Scholar

  • Rabiner, L. (1989). A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286. 10.1109/5.18626 First citation in articleCrossref, Google Scholar

  • Richters, C., Stadler, M., Radkowitsch, A., Schmidmaier, R., Fischer, M. R., & Fischer, F. (2023). Who is on the right track? Behavior-Based prediction of diagnostic success in a collaborative diagnostic reasoning simulation. Large-Scale Assessments in Education, 11(1), Article 3. 10.1186/s40536-023-00151-1 First citation in articleCrossref, Google Scholar

  • Schwichow, M., Croker, S., Zimmerman, C., Höffler, T., & Härtig, H. (2016). Teaching the control-of-variables strategy: A meta-analysis. Developmental Review, 39, 37–63. 10.1016/j.dr.2015.12.001 First citation in articleCrossref, Google Scholar

  • Sommerhoff, D., Codreanu, E., Nickl, M., Ufer, S., & Seidel, T. (2023). Pre-service teachers’ learning of diagnostic skills in a video-based simulation: Effects of conceptual vs. interconnecting prompts on judgment accuracy and the diagnostic process. Learning and Instruction, 83, Article 101689. 10.1016/j.learninstruc.2022.101689 First citation in articleCrossref, Google Scholar

  • Stadler, M., Fischer, F., & Greiff, S. (2019). Taking a closer look: An exploratory analysis of successful and unsuccessful strategy use in complex problems. Frontiers in Psychology, 10, Article 777. 10.3389/fpsyg.2019.00777 First citation in articleCrossref, Google Scholar

  • Stadler, M., Iliescu, D., & Greiff, S. (2021). Validly authentic. European Journal of Psychological Assessment, 37(6), 419–422. 10.1027/1015-5759/a000686 First citation in articleLink, Google Scholar

  • Stadler, M., Niepel, C., & Greiff, S. (2016). Easily too difficult: Estimating item difficulty in computer simulated microworlds. Computers in Human Behavior, 65, 100–106. 10.1016/j.chb.2016.08.025 First citation in articleCrossref, Google Scholar

  • Stadler, M., Pickal, A. J., Brandl, L., & Krieger, F. (2024). HMM – epistemic activities CPS [Data]. https://osf.io/vs4fw First citation in articleGoogle Scholar

  • Stender, A., Schwichow, M., Zimmerman, C., & Härtig, H. (2018). Making inquiry-based science learning visible: The influence of CVS and cognitive skills on content knowledge learning in guided inquiry. International Journal of Science Education, 40(15), 1812–1831. 10.1080/09500693.2018.1504346 First citation in articleCrossref, Google Scholar

  • Tang, X., Wang, Z., He, Q., Liu, J., & Ying, Z. (2020). Latent feature extraction for process data via multidimensional scaling. Psychometrika, 85(2), 378–397. 10.1007/s11336-020-09708-3 First citation in articleCrossref, Google Scholar

  • Tschirgi, J. E. (1980). Sensible reasoning: A hypothesis about hypotheses. Child Development, 51(1), Article 1. 10.2307/1129583 First citation in articleCrossref, Google Scholar

  • Wong, J., Baars, M., de Koning, B. B., van der Zee, T., Davis, D., Khalil, M., Houben, G.-J., & Paas, F. (2019). Educational theories and learning analytics: From data to knowledge. In D. IfenthalerD.-K. MahJ. Y.-K. Yau (Eds.), Utilizing learning Analytics to support study success (pp. 3–25). Springer International Publishing. 10.1007/978-3-319-64792-0_1 First citation in articleCrossref, Google Scholar

  • Wüstenberg, S., Greiff, S., & Funke, J. (2012). Complex problem solving—More than reasoning? Intelligence, 40(1), 1–14. 10.1016/j.intell.2011.11.003 First citation in articleCrossref, Google Scholar

  • Zimmerman, C. (2007). The development of scientific thinking skills in elementary and middle school. Developmental Review, 27(2), 172–223. 10.1016/j.dr.2006.12.001 First citation in articleCrossref, Google Scholar