Patterns of Multilevel Variance in Psycho-Educational Phenomena: Comparing Motivation, Engagement, Climate, Teaching, and Achievement Factors
Abstract
Abstract. The present study explored multilevel variance for a range of salient psycho-educational factors in mathematics. With a sample of 4,383 students (Years 5–8) in 257 classrooms and 47 schools, data indicated patterns of variance across the selection of psycho-educational factors. For all factors, the bulk of variance resided at the student (and residual) level. In ascending order of upper-level variance were motivation, perceived motivational ‘climate’, homework completion, teacher-student relationships, and achievement – with motivation and perceived ‘climate’ yielding very little upper-level variance. Hence, although there is usually a hierarchical structure in which psycho-educational factors are situated, there is variation in patterns of multilevel variance across the range of factors. In exploring a range of psycho-educational phenomena from a multilevel perspective, the present study offers further direction for researchers selecting and operationalizing psycho-educational phenomena in multilevel research. Implications for pedagogy, classroom climate, and separating teaching effects from teacher effects and schooling effects from school effects are also discussed.
Zusammenfassung. Die vorliegende Studie untersucht für bedeutsame pädagogisch-psychologische Variablen im Fach Mathematik den Anteil an Varianz, der auf eine höhere Ebene (Klasse, Schule) zurückgeht. Die Daten einer Stichprobe von N = 4383 Schülerinnen und Schülern im Alter zwischen 5 und 8 Jahren aus 257 Klassen und 47 Schulen wurden analysiert. Es zeigten sich verschiedene Muster von Varianzanteilen auf den unterschiedlichen Analyseebenen. Für alle Variablen zeigte sich, dass der Großteil der Varianz auf Schülerebene (und beim Residuum) angesiedelt war. Die Faktoren Motivation, wahrgenommenes motivationales Klima, Fertigstellung von Hausaufgaben, Lehrer-Schüler-Beziehung und Leistung lassen sich hinsichtlich der Varianz, die auf eine höhere Ebene zurückgeht, in aufsteigender Reihenfolge ordnen, wobei sich für Motivation und das wahrgenommene Klima nur ein sehr geringer Anteil Varianz auf Ebene 2 ergeben hat. Obwohl pädagogisch-psychologische Daten also in der Regel hierarchisch strukturiert sind, gibt es Unterschiede in den Mustern der mehrebenenanalytischen Varianz für verschiedene Faktoren. Die vorliegende Studie unterstützt Wissenschaftlerinnen und Wissenschaftler bei der Auswahl und Operationalisierung pädagogisch-psychologischer Phänomene, indem mehrere pädagogisch-psychologische Phänomene mehrebenenanalytisch untersucht wurden. Pädagogische Implikationen, das Klassenklima, sowie die Trennung von instruktionalen Effekten und Effekten der Lehrkraft und die Trennung von Effekten der Beschulung und der Schule werden diskutiert.
Literatur
(1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology , 84, 261–271.
(1997). Self-efficacy: The exercise of control. New York: Freeman & Co.
(2008). Cross-classified random effects models. In A. A. O’Connell & D. B. McCoach (Eds). Multilevel modeling of educational data . (pp. 161–197). Charlotte, NC: Information Age Publishing.
(1979). The ecology of human development: Experiments by nature and design . Cambridge, MA: Harvard University Press.
(1992). Ecological systems theory. In R. Vasta (Ed). Six theories of child development: Revised reformulations and current issues (pp. 187–249). London: Jessica Kingsley Pub.
(2001). The bioecological theory of human development. In N. J. Smelser & P. B. Baltes (Eds). International encyclopedia of the social and behavioral sciences (Vol 10, pp. 6963–6970). New York: Elsevier.
(1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology , 83, 234–246.
(2007). Sensitivity of goodness of fit indices to lack of measurement invariance. Structural Equation Modeling , 14, 464–504.
(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling , 9, 233–255.
(2010). Homework works if homework quality is high: Using multilevel modeling to predict the development of achievement in mathematics. Journal of Educational Psychology , 102, 467–482.
(2008). Determinants of classroom environment in Queensland secondary schools: A multilevel reanalysis. Educational Research and Evaluation , 14, 429–444.
(2007). Perceived learning environment and students’ emotional experiences: A multilevel analysis of mathematics classrooms. Learning and Instruction , 17, 478–493.
(2003). Multilevel statistical models (3rd ed.). London: Hodder Arnold.
(2009). Multilevel modeling and school psychology: A review and practical example. School Psychology Quarterly , 24, 84–94.
(2009). Visible learning . Oxford: Routledge.
(2001). Multilevel modelling with SEM. In G. A. Marcoulides & R. E. Schumacker (Eds). New developments and techniques in structural equation modelling . (pp. 89–127). Mahwah, NJ: Lawrence Erlbaum Assoc.
(2003). Interpreting the magnitudes of correlation coefficients. American Psychologist , 58, 78–80.
(2008). Modeling growth using multilevel and alternative approaches. In A. A. O’Connell & D. B. McCoach (Eds). Multilevel modeling of educational data . (pp. 111–159). Charlotte, NC: Information Age Publishing.
(2010). Multilevel analysis: Techniques and applications. New York: Routledge.
(2006). LISREL 8.80. Chicago: Scientific Software International.
(2005). Attitudes, intentions, and participation. Longitudinal Surveys of Australian Youth Research Report 41. Melbourne: Australian Council for Educational Research.
(1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement , 56, 746–759.
(1994). Levels issues in theory development, data collection, and analysis. The Academy of Management Review , 19, 195–229.
(2008). A multilevel study of predictors of student perceptions of school climate: The effect of classroom-level factors. Journal of Educational Psychology , 100, 96–104.
(2007). Effective classroom management and the development of subject-related interest. Learning and Instruction , 17, 494–509.
(2008). Using multilevel modeling to investigate school effects. In A. A. O’Connell & D. B. McCoach (Eds). Multilevel modeling of educational data . (pp. 59–110). Charlotte, NC: Information Age Publishing.
(2008). A multilevel perspective on gender in classroom motivation and climate: Potential benefits of male teachers for boys? Journal of Educational Psychology , 100, 78–95.
(2008). Social comparison and big-fish-little-pond effects on self-concept and other self-belief constructs: Role of generalized and specific others. Journal of Educational Psychology , 100, 3, 510–524.
(2008). The Motivation and Engagement Scale . Sydney: Lifelong Achievement Group (www.lifelongachievement.com).
(2009). Motivation and engagement across the academic lifespan: A developmental construct validity study of elementary school, high school, and university/college students. Educational and Psychological Measurement , 69, 794–824.
(2010). Should students have a gap year? Motivation and performance factors relevant to time out after completing school. Journal of Educational Psychology , 102, 561–576.
(2009). Interpersonal relationships, motivation, engagement, and achievement: Yields for theory, current issues, and practice. Review of Educational Research , 79, 327–365.
(2011). Quantitative modelling of correlational and multilevel data in educational research: A construct validity approach to exploring and testing theory. In L. Markauskaite, P. Freebody, & J. Irwin (Eds.). Methodological choices and research designs for educational and social change: Linking scholarship, policy and practice . Dordrecht: Springer.
(2010). Multilevel motivation and engagement: Assessing construct validity across students and schools. Educational and Psychological Measurement , 70, 973–989.
(1990). Choosing a multivariate model: Noncentrality and goodness-of-fit. Psychological Bulletin , 107, 247–255.
(1997). Patterns of Adaptive Learning (PALS) . Ann Arbor, MI: University of Michigan.
(1999). The structure and function of collective constructs: Implications for multilevel research and theory development. The Academy of Management Review , 24, 249–265.
(1998). Levels of analysis issues in group psychology: Using efficacy as an example of a multilevel model. Group Dynamics: Theory, Research, and Practice , 2, 285–298.
(2007). Mplus user’s guide (5th ed.). Los Angeles, CA: Mplus.
(2008). (Eds). Multilevel modeling of educational data . Charlotte, NC: Information Age Publishing.
(1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ) (1991). Ann Arbor, MI: National Center for Research to Improve Postsecondary Teaching and Learning.
(2010). A user’s guide to MLwiN version 2.18 . Institute of Education, University of London.
(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
(2000). School as a context of early adolescents’ academic and social-emotional development: A summary of research findings. Elementary School Journal , 100, 443–471.
(1996). A beginner’s guide to structural equation modeling . Mahwah, NJ: Erlbaum.
(2007). Students’ self-reported effort and time on homework in six school subjects: Between-students differences and within-student variation. Journal of Educational Psychology , 99, 432–444.
(2009). Predicting homework motivation and homework effort in six school subjects: The role of person and family characteristics, classroom factors, and school track. Learning and Instruction , 19, 243–258.
(2009). Between-teacher differences in homework assignments and the development of students’ homework effort, homework emotions, and achievement. Journal of Educational Psychology , 101, 176–189.
(2002). Adolescents’ expectancies for success and achievement task values during middle and high school years. In F. Pajares & T. Urdan (Eds). Academic motivation of adolescents . Connecticut: Information Age Publishing.
(1993). Wide Range Achievement Test . Wilmington, DE: Wide Range Inc.