Predictors of teachers' appreciation of evidence-based practice and educational research findings
Abstract
Abstract. Teachers' reception of educational research is considered important for improving teaching and student learning. Yet, it is a challenging task requiring teachers to have access to scientific sources, the skill and time to find and exhaust such resources, and the capacity to interpret retrieved information. If such essential conditions are not met, teachers have hardly any chance to engage in research reception and, consequently, may question the value and relevance of research findings to their practice. Prior research has suggested that teachers are indeed critical of educational research findings and rarely refer to them. Based on data from the field trial (N = 674) and main study (N = 2,549) of a national extension study of the Programme for International Student Assessment (PISA) 2012 in Germany, this study explored the role of (a) teachers' access to scientific sources, (b) perceived lack of skill and time to search for research findings, and (c) their familiarity with research methods/statistics as potential predictors of their appreciation of evidence-based practice, and perceived irrelevance of educational research findings. Structural equation models demonstrated that perceived lack of skill and time to find research findings, in particular, substantially affected participants' irrelevance perceptions. The more participants assessed their sourcing skill and time to be too constrained to engage in research reception, the more they judged research findings to be irrelevant to their practice. Though source access and familiarity with research methods/statistics indicated only small or even no effects, they strongly correlated with participants' perceived lack of sourcing skill and time. Better source access and greater familiarity were associated with less concern about one's skill and time resources to search for relevant research findings. These findings potentially underline the relevance of strengthening both teachers' access to scientific sources and individual capacities to understanding research contents.
Zusammenfassung. Die Rezeption bildungswissenschaftlicher Befunde soll Lehrkräften zur Verbesserung ihrer Unterrichtspraxis dienen. Dies stellt eine anspruchsvolle Aufgabe dar, die es erfordert, dass Lehrkräfte Zugang zu Forschungsliteratur haben, über die nötigen Fähigkeiten und Zeitressourcen zur Rezeption verfügen, und im Stande sind, solche Befunde zu interpretieren. Sind solche essentiellen Voraussetzungen nicht erfüllt, haben Lehrkräfte kaum Chancen, bildungswissenschaftliche Befunde heranzuziehen und zu rezipieren. Dies kann dazu führen, dass sie bildungswissenschaftlichen Befunden und evidenzbasiertem Handeln weniger Wert und Relevanz für ihr Handeln beimessen. Vorangegangene Untersuchungen deuten tatsächlich darauf hin, dass Lehrkräfte die Relevanz bildungswissenschaftlicher Befunde kritisch betrachten und diese kaum nutzen. Basierend auf Daten von Mathematiklehrkräften, die an der Befragung der nationalen Erweiterungsstudie zur PISA Studie 2012 (Deutschland) im Feldversuch (N = 674) sowie in der Hauptstudie (N = 2549) teilnahmen, untersucht dieser Beitrag, welchen Einfluss (a) der Zugang zu Forschungsliteratur, (b) die Wahrnehmung eigener Fähigkeiten und Zeitressourcen, Forschungsergebnisse finden zu können, sowie (c) die Vertrautheit mit Forschungsmethoden und Statistik auf die Wertschätzung von evidenzbasierter Praxis einerseits und auf die wahrgenommene Irrelevanz von bildungswissenschaftlichen Befunden andererseits haben. Analysen mittels Strukturgleichungsmodellen ergaben, dass die Beurteilung bildungswissenschaftlicher Befunde als irrelevant insbesondere durch die Wahrnehmung unzureichender Fähigkeiten und Zeitressourcen vorhergesagt wurde. Je mehr die Lehrkräfte ihre Fähigkeit und Zeit als unzureichend einschätzten, desto stärker lehnten sie bildungswissenschaftliche Befunde auch als irrelevant ab. Der Zugang zu Forschungsliteratur bzw. die Vertrautheit mit Forschungsmethoden und Statistik hatten nur kleine, teils auch keine, Effekte auf die Wahrnehmung evidenzbasierten Handelns und bildungswissenschaftlicher Befunde. Allerdings korrelierten diese beiden Prädiktoren hoch mit der Wahrnehmung der eigenen Fähigkeiten und Zeitressourcen. Je besser der Zugang zu Forschungsliteratur bzw. je höher die Vertrautheit mit Forschungsmethoden und Statistik angegeben wurde, desto weniger äußerten Lehrkräfte Bedenken hinsichtlich ihrer Fähigkeiten und Zeitressourcen, Forschungsbefunde finden und lesen zu können. Diese Ergebnisse untermauern die Relevanz, den Zugang zu Forschungsliteratur sowie auch grundlegende Fähigkeiten zur Wissenschaftsrezeption zu stärken.
References
2010). Weighted least squares estimation with missing data . Available from http://www.statmodel.com/download/GstrucMissingRevision.pdf
(2017). Wie können Lehrkräfte und ihre Schülerinnen und Schüler lernen, fragile Evidenz zu verstehen und zu nutzen? [How can teachers and students learn to understand and use fragile evidence?]. Psychologische Rundschau , 68 (3), 188–192. https://doi.org/10.1026/0033-3042/a000363
(2012). European teacher training reforms. Science , 336 , 1642–1643. https://doi.org/10.1126/science.1218387
(2015). Evidenzbasierte Praxis – im Lehrerberuf?! Einführung in den Thementeil [Evidence-based practice – In teaching?! Introduction to the Special Issue]. Unterrichtswissenschaft , 43 (3), 188–192. https://doi.org/10.3262/UW1503188
(2018). Introduction to research on multiple source use. In J. L. Bråten, & M. T. McCrudden (Eds.), Handbook of multiple source use (pp.1–13). New York: Routledge. https://doi.org/10.4324/9781315627496
(2007). The gap between educational research and practice: A literature review, symposium, and questionnaire. Educational Research and Evaluation , 13 (3), 203–220. doi.org/10.1080/13803610701626127
(2014). Empirische Bildungsforschung und evidenzbasierte Bildungspolitik [Empirical educational research and evidence-based educational policy]. Zeitschrift für Erziehungswissenschaft , 17 (4), 3–54. https://doi.org/10.1007/s11618-014-0514-5
(2018).
(The provenance of certainty: Multiple source use and the public engagement with science . In J. L. G. Braasch I. Bråten M. T. McCrudden (Eds.), Handbook of multiple source use (pp.269–284). New York, NY: Routledge.2015). Confirmatory factor analysis for applied research . New York: Guilford.
(2017). Combining the best of two worlds: A conceptual proposal for evidence-informed school improvement. Educational Research , 59 (2), 154–172. https://doi.org/10.1080/00131881.2017.1304327
(2016). Is engaging in evidence-informed practice in education rational? What accounts for discrepancies in teachers' attitudes towards evidence use and actual instances of evidence use in schools? British Educational Research Journal , 42 (5), 780–801. https://doi.org/10.1002/berj.3239
(2016). Research utilisation and the struggle for the teacher's soul: A narrative review. European Journal of Teacher Education , 39 (5), 616–629. https://doi.org/10.1080/02619768.2016.1252912
(2016). Mobilising research knowledge for teaching and teacher education. European Journal of Teacher Education , 39 (5), 529–533. https://doi.org/10.1080/02619768.2016.1256086
(1988). Statistical power analysis for the behavioral sciences (2nd ed.) . Hillsdale, NJ: Lawrence Erlbaum Associates.
(2000). Predictors of educators' valuing of systematic inquiry in schools. Canadian Journal of Program Evaluation , 25–52.
(2012). Use of research-based information by school practitioners and determinants of use: A review of empirical research. Evidence & Policy: A Journal of Research, Debate and Practice , 8 (3), 285–309. https://doi.org/10.1332/174426412X654031
(Eds.). (2005). Preparing teachers for a changing world: What teachers should learn and be able to do . San Francisco, CA: Jossey-Bass.
(1999). What is evidence-based education? British Journal of Educational Studies , 47 (2), 108–121. https://doi.org/10.1111/1467-8527.00106
(Department for Education . (2016). Standards for teachers' professional development . https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/537030/160712_-_PD_standard.pdf2020). Evidence-based practice in higher education: Teacher educators' attitudes, challenges, and uses. Frontiers in Education , 5 (62). https://doi.org/10.3389/feduc.2020.00062
(2009). Preparing information literate teachers: A metasynthesis. Library & Information Science Research , 31 (4), 247–256. https://doi.org/10.1016/j.lisr.2009.04.003
(1996).
(Gender development and gender effects . In D. C. Berliner R. C. Calfee (Eds.), Handbook of educational psychology (pp.358–396). Macmillan Library Reference USA.European Commission [EC] . (2007). Towards more knowledge-based policy and practice in education and training . Available from https://op.europa.eu/en/publication-detail/-/publication/962e3b89-c546-4680-ac84-777f8f10c590/language-en2018). Rethinking connections between research and practice in education: A conceptual framework. Educational Researcher , 47 (4), 235–245. https://doi.org/10.3102/0013189X18761042
(2006). Forming attitudes that predict future behavior: A meta-analysis of the attitude-behavior relation. Psychological Bulletin , 132 (5), 778–822. https://doi.org/10.1037/0033-2909.132.5.778
(2004). Visioning the academic–teacher divide: Power and knowledge in the educational community. Teachers and Teaching , 10 (1), 35–58. https://doi.org/10.1080/13540600320000170918
(2000).
(Teaching as a research-based profession: Possibilities and prospects . In B. Moon (Ed.), Leading professional development in education (pp.200–210). London: Routledge.2011). Visible learning for teachers: Maximizing impact on learning . London: Routledge.
(2015). An introduction to multilevel modeling techniques ( 3rd ed. ). London: Routledge.
(2003). The use of research to improve professional practice: A systematic review of the literature. Oxford Review of Education , 29 (4), 449–471. https://doi.org/10.1080/0305498032000153025
(2015). Wozu nutzen Lehrkräfte welche Ressourcen? Eine Interviewstudie zur Schnittstelle zwischen bildungswissenschaftlicher Forschung und professionellem Handeln im Bildungsbereich [For what do teachers use which kind of resources? An interview study on the interface between research and professional practice in education]. Unterrichtswissenschaft , 43 (3), 193–208.
, (2003). Evidence-based practice: Beliefs, attitudes, knowledge, and behaviors of physical therapists. Journal of the American Physical Therapy , 83 (9), 786–805. https://doi.org/10.1093/ptj/83.9.786
, (2003). The development and validation of a knowledge, attitude and behaviour questionnaire to assess undergraduate evidence-based practice teaching and learning. Medical Education , 37 (11), 992–1000. https://doi.org/10.1046/j.1365-2923.2003.01678.x
(2020). What influences teachers' “buy-in” of research? Teachers' beliefs about the applicability of educational research to their practice. Teaching and Teacher Education , 88 , Article 102980. https://doi.org/10.1016/j.tate.2019.102980.
(Kultusministerkonferenz . (2004). Standards der Lehrerbildung: Bildungswissenschaften [Standards of teacher education: Educational sciences]. Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany, Berlin . Available from https://www.kmk.org/fileadmin/veroeffentlichungen_beschluesse/2004/2004_12_16-Standards-Lehrerbildung.pdf2009). The use of evidence-based instructional strategies in special education settings in secondary schools: Development, implementation and outcomes. Teaching and Teacher Education , 25 , 917–923. https://doi.org/10.1016/j.tate.2009.02.016
(2003). Factors affecting teachers' participation in professional learning activities. Teaching and Teacher Education , 19 , 149–170. https://doi.org/10.1016/S0742-051X(02)00101-4
(2011). Mobilising research knowledge in education. London Review of Education , 9 (1), 15–16. https://doi.org/10.1080/14748460.2011.550431
(2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling , 9 (2), 151–173. https://doi.org/10.1207/S15328007SEM0902_1
(2014). Educational research in educational practice: Predictors of use. Canadian Journal of Education , 37 (2), 1–26.
(2018). PISA 2012 Skalenhandbuch. Dokumentation der Erhebungsinstrumente [PISA 2012 documentation of measurement instruments] . Münster: Waxmann.
, (2010).
(Academic self-concept: The role of positive self-beliefs in educational psychology . In K. Littleton C. Wood J. Kleine Staarman (Eds.), International handbook of psychology in education (pp.499–534). Bingley, UK: Emerald Group Publishing.2012). “Doing research was inspiring”: Building a research community with teachers. Educational Action Research , 20 (3), 385–406. https://doi.org/10.1080/09650792.2012.697402
(2017). Pre-service teachers' perceived value of general pedagogical knowledge for practice: Relations with epistemic beliefs and source beliefs. PloS ONE , 12 (9), e0184971. https://doi.org/10.1371/journal.pone.0184971
(2017). Mplus User's Guide (8th ed.). Los Angeles, CA: Muthén & Muthén.
(2008). Research-based teacher education for teachers' lifelong learning. Lifelong Learning in Europe , 13 (1), 61–69.
(2003). From knowing to doing: A framework for understanding the evidence-into-practice agenda. Evaluation , 9 (2), 125–148. https://doi.org/10.1177/1356389003009002002
(Organisation for Economic Cooperation and Development [OECD] . (2005). Teachers matter . Paris: OECD Publishing.2009). Evidence-based teaching: A practical approach . Cheltenham: Nelson Thornes.
(Eds.). (2013). PISA 2012. Fortschritte und Herausforderungen in Deutschland [ PISA 2012. Improvements and challenges in Germany ]. Münster: Waxmann.
(2003). Validation of the Fresno test of competence in evidence based medicine. BMJ: British Medical Journal , 326 , 319–321. https://doi.org/10.1136/bmj.326.7384.319
(2005). Evidence-based practice in science education: The researcher – user interface. Research Papers in Education , 20 (2), 169–186. https://doi.org/10.1080/02671520500078036
, (2011). Professional development across the teaching career: Teachers' uptake of formal and informal learning opportunities. Teaching and Teacher Education , 27 , 116–126. https://doi.org/10.1016/j.tate.2010.07.008
(2016). Evidence-based practice: The psychology of EBP implementation. Annual Review of Psychology , 67 , 667–692. https://doi.org/10.1146/annurev-psych-122414-033336
(2008). Practitioner's guide to using research for evidence-based practice . Hoboken, N.J.: Wiley.
(1996). Evidence-based medicine: What it is and what it isn't. BMJ: British Medical Journal , 312 , 71–72. https://doi.org/10.1136/bmj.312.7023.71
(2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education , 26 (3), 482–496. https://doi.org/ 10.1016/j.tate.2009.06.007
(2019). TALIS 2018. Rahmenbedingungen des schulischen Lehrens und Lernens aus Sicht von Lehrkräften und Schulleitungen im internationalen Vergleich (Bd. 1) [Teachers’ and school leaders' perspectives on conditions of teaching and learning in schools]. Graz: Leykam.
(2017). Meta-Analysen zur Unterrichtsforschung und ihr Beitrag für die Realisierung eines Clearing House Unterricht für die Lehrerbildung [Meta-analyses of classroom research and their contribution to the realization of a clearinghouse initiative for teacher education]. Zeitschrift für Bildungsforschung , 7 (3), 311–325. https://doi.org/10.1007/s35834-017-0191-6
(2006). Instruments for evaluating education in evidence-based practice: a systematic review. JAMA , 296 (9), 1116–1127. https://doi.org/10.1001/jama.296.9.1116
, (2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher , 31 (7), 15–21. https://doi.org/10.3102/0013189X031007015
(2020). How evidence-based reform will transform research and practice in education. Educational Psychologist , 55 (1), 21–31. https://doi.org/10.1080/00461520.2019.1611432
(2017). Probleme evidenzbasierter bzw.-orientierter pädagogischer Praxis [Problems of evidence-based or rather evidence-oriented educational practice]. Zeitschrift für Pädagogische Psychologie , 31 (2), 99–110. https://doi.org/10.1024/1010-0652/a000201
(2016). How source information shapes lay interpretations of science conflicts: Interplay between sourcing, conflict explanation, source evaluation, and claim evaluation. Reading and Writing , 29 (8), 1629–1652. https://doi.org/10.1007/s11145-016-9638-8
(2015). Nutzung von Evidenz im Bildungsbereich. Validierung eines Instruments zur Erfassung von Kompetenzen der Informationswahl und Bewertung von Studien [Using evidence in education. Validating an isntrument for measureing competencies of information selection and evalution of research studies]. Zeitschrift für Pädagogik , 61 , 144–166.
(2010). The gap between educational research and practice: Views of teachers, school leaders, intermediaries and researchers. British Educational Research Journal , 36 (2), 299–316. https://doi.org/10.1080/01411920902919257
(2018). Barriers and conditions for teachers' utilisation of academic knowledge. International Journal of Educational Research , 90 , 50–63. https://doi.org/10.1016/j.ijer.2018.05.003
(2013). Der Nutzen der forschungsmethodischen Ausbildung für angehende Lehrpersonen [The benefits pre-service teachers derive from tuition in research methodology]. Beiträge zur Lehrerinnen-und Lehrerbildung , 31 (3), 321–332.
(What Works Clearinghouse . (2020). WWC standards handbook , version 4.1. Available from https://ies.ed.gov/ncee/wwc/Docs/referenceresources/WWC-Standards-Handbook-v4-1-508.pdf2015). Kompetenz angehender Lehrkräfte zum Argumentieren mit Evidenz: Erhöht ein Training von Heuristiken die Argumentationsqualität? [Pre-service teachers' evidence-based argumentation competence: Can a training of heuristics improve argumentative quality?]. Unterrichtswissenschaft , 43 (3), 209–224. https://doi.org/ 10.3262/UW1503209
(2007). Teachers' approaches to finding and using research evidence: An information literacy perspective. Educational Research , 49 (2), 185–206. https://doi.org/10.1080/00131880701369719
(2001). Evidence-based medicine in general practice: Beliefs and barriers among Australian GPs. Journal of Evaluation in Clinical Practice , 7 (2), 201–210. https://doi.org/10.1046/j.1365-2753.2001.00294.x
(