Abstract
Der Einsatz von Fragebögen zur Lehrevaluation gehört an vielen Hochschulen zum gängigen Vorgehen im Rahmen der Qualitätssicherung. Dabei werden nicht immer psychometrisch gut fundierte Fragebögen eingesetzt. Dieser Artikel schildert ein Praxisbeispiel, das verdeutlicht, wie anhand bestehender Fragebögen und durch den Einsatz qualitativer und quantitativer Methoden ein facettiertes Lehrevaluationsinventar konstruiert und evaluiert wurde. Dabei wird aufgezeigt, wie anhand der Critical Incident Technique ein gemeinsames Verständnis aller Beteiligten zur Bedeutung guter Lehre definiert werden kann. Darüber hinaus wird das Berliner Lehrevaluationsinventar für Vorlesungen (BLEI-VL) vorgestellt und hinsichtlich seiner faktoriellen Validität geprüft. Das BLEI-VL spiegelt das gemeinsame Verständnis guter Lehre von Dozierenden und Studierenden wieder und erlaubt die Erfassung der Qualität der Lehre bezüglich der Aspekte Vorbereitung der Vorlesung, Didaktik und Interaktion.
Many universities regularly use teaching evaluation questionnaires in the context of quality management. However, sometimes the questionnaires used are not psychometrically tested. This paper details a practical example which shows how to combine existing questionnaires, qualitative and quantitative methods to create and test a facetted teaching evaluation inventory. It is shown how the critical incident technique was used to define a common understanding of good teaching. Moreover, the Berlin Teaching Evaluation Inventory for Lectures (BTEI-L) is introduced and tested with regard to its factorial validity. This inventory reflects the common understanding of good teaching and allows the assessment of teaching quality for preparation of lectures, didactics and interaction.
Literatur
(2007). The dimensionality of student ratings of instruction: What we know and what we do not. In R. Perry & J. SmartEds. The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 385 – 456). Dordrecht: Springer.
(2007). Instructors’ perspectives on the utility of student ratings of instruction. Instructional Science, 37, 171 – 184.
(2003). Hängen Ergebnisse einer Lehrveranstaltungs-Evaluation von der Häufigkeit des Veranstaltungsbesuches ab? Zeitschrift fur Padagogische Psychologie, 17, 125 – 131.
(2005). Fifty years of the critical incident technique: 1954 – 2004 and beyond. Qualitative Research, 5, 475 – 497.
(1966). The Scree Test for the number of factors. Multivariate Behavioral Research, 1, 245 – 276.
(2003). Student evaluation of teaching effectiveness. Assessment & Evaluation in Higher Education, 28, 71 – 88.
(2009). Student evaluations of teaching: Are they related to what students learn? A meta-analysis and review of the literature. Journal of Marketing Education, 31, 16 – 30.
(1990). Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative sociology, 13, 3 – 21.
(2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13, 230 – 253.
(1954). The critical incident technique. Psychological Bulletin, 51, 327 – 358.
(2008). Moving towards multidimensional evaluation of teaching in higher education: A study across four faculties. Higher Education, 56, 583 – 597.
(2011). Masking misfit in confirmatory factor analysis by increasing unique variances: A cautionary note on the usefulness of cutoff values of fit indices. Psychological Methods, 16, 319 – 336.
(1965). An empirical comparison of methods for estimating factor scores. Educational and Psychological Measurement, 25, 313 – 322.
(2000). A comprehensive approach to the evaluation of college teaching. New Directions for Teaching and Learning, 83, 109 – 123.
(1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11, 253 – 388.
(1991). Multidimensional students evaluations of teaching effectiveness–a test of alternative higher-order structures. Journal of Educational Psychology, 83, 285 – 296.
(2007). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In R. P. Perry and J. C. SmartEds., The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 319 – 383). Dordrecht: Springer.
(2011). Use of student ratings to benchmark universities: Multilevel modeling of responses to the Australian Course Experience Questionnaire (CEQ). Journal of Educational Psychology, 103, 733.
(1999). Test Theory. Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
(2009). Moving towards multidimensional evaluation of teaching in higher education: A study across four faculties. Quality & Quantity, 43, 197 – 209.
(2001). Faculty thoughts and concerns about student ratings. New Directions for Teaching and Learning, 87, 3 – 15.
(2008). Profiling teacher/teaching using descriptors derived from qualitative feedback: Formative and summative applications. Research in Higher Education, 50, 73 – 100.
(2003). Changing the agenda for research into students views about university teaching: Four shortcomings of SRT research. Teaching in Higher Education, 8, 399 – 411.
(2014). psych: Procedures for Personality and Psychological Research. 1.4.5. Available at CRAN.R-project.org/package=psych
(2005). Instruments for obtaining student feedback: a review of the literature. Assessment & Evaluation in Higher Education, 30, 387 – 415.
(2003). Lehrevaluation an Hochschulen: Schlussfolgerungen aus Forschung und Anwendung für Hochschulunterricht und seine Evaluation. Zeitschrift für Evaluation, 2, 233 – 256.
(2000). Student evaluations of teaching: An exploratory study of the faculty response. Journal of Marketing Education, 22, 199 – 213.
(2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83, 598 – 642.
(2005). A multilevel factor analysis of students’ evaluations of teaching. Educational and Psychological Measurement, 65, 272 – 296.
(1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41, 321‐327.
(1995). Student evaluation of teaching: Some cautions and suggestions. Teaching Sociology, 23, 64 – 68.
(2005). Cronbach’s α, Revelle’s β, and McDonald’s ωH: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70, 123 – 133.
(2006). Estimating generalizability to a latent variable common to all of a scale’s indicators: A comparison of estimators for ωh. Applied Psychological Measurement, 30, 121 – 144.
