Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article

Measurement Invariance of Children’s SEL Competencies

An Examination of the SSIS SEL Brief Scales With a Multi-Informant Sample From Six Countries

Published Online:January 31, 2023https://doi.org/10.1027/1015-5759/a000753

Abstract

Abstract. Although children use social and emotional learning skills (SEL) across the world, the expression of these skills may vary across cultures and developmental levels. Such variability complicates the process of assessing SEL competencies with consequences for understanding differences in SEL skills and developing interventions. To address these challenges, the current study examined the measurement invariance of translated versions of a brief, multi-informant (Teacher, Parent, Student) measure of SEL skills developed in the US with data from six European countries (Croatia, Greece, Italy, Latvia, Portugal, and Romania; n = 10,602; 8,520; 6,611, for the SSIS SELb – Teacher, Parent, and Student versions, respectively). In addition to cross-country invariance testing, we conducted measurement invariance testing across ages (Primary and Secondary students) for the Teacher and Student forms of the measure. Results revealed a high degree of measurement invariance across countries (Scalar for the Teacher form and Partial Scalar for the Parent and Student form) and developmental levels (Scalar for the Teacher form and Partial Scalar for the Student form), supporting the use of translated versions of the SSIS SELb for international research across these countries and developmental levels. Implications are discussed for assessment and promoting children’s SEL competencies globally.

References

  • Anthony, C. J., Brann, K., Elliott, S. N., & Garis, E. J. (2022). Examining the structural validity of the SSIS SEL Brief Scales – Teacher and Student Forms. Psychology in the Schools, 59, 260–280. https://doi.org/10.1002/pits.22607 First citation in articleCrossref, Google Scholar

  • Anthony, C. J., DiPerna, J. C., & Lei, P. W. (2016). Maximizing measurement efficiency of behavior rating scales using Item Response Theory: An example with the Social Skills Improvement System–Teacher Rating Scale. Journal of School Psychology, 55, 57–69. https://doi.org/10.1016/j.jsp.2015.12.005 First citation in articleCrossref, Google Scholar

  • Anthony, C. J., Elliott, S. N., DiPerna, J. C., & Lei, P.-W. (2020a). The SSIS SEL Brief Scales – Student Form: Initial development and validation. School Psychology, 35(4), 277–283. https://doi.org/10.1037/spq0000390 First citation in articleCrossref, Google Scholar

  • Anthony, C. J., Elliott, S. N., DiPerna, J. C., & Lei, P. W. (2020b). Multirater assessment of young children’s social and emotional learning via the SSIS SEL Brief Scales – Preschool Forms. Early Childhood Research Quarterly, 53, 625–637. https://doi.org/10.1016/j.ecresq.2020.07.006 First citation in articleCrossref, Google Scholar

  • Anthony, C. J., Elliott, S. N., DiPerna, J. C., & Lei, P.-W. (2021). Initial development and validation of the SSIS SEL Brief Scales – Teacher Form. Journal of Psychoeducational Assessment, 39(2), 166–181. https://doi.org/10.1177/0734282920953240 First citation in articleCrossref, Google Scholar

  • Anthony, C. J., Elliott, S. N., Yost, M., Lei, P.-W., DiPerna, J. C., Cefai, C., Camilleri, L., Bartolo, P. A., Grazzani, I., Ornaghi, V., Cavioni, V., Conte, E., Vorkapić, S. T., Poulou, M., Martinsone, B., Simões, C., & Colomeischi, A. A. (2022). Multi-informant validity evidence for the SSIS SEL Brief Scales across six European countries. Frontiers in Psychology, 13, Article 928189. https://doi.org/10.3389/fpsyg.2022.928189 First citation in articleCrossref, Google Scholar

  • Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834 First citation in articleCrossref, Google Scholar

  • Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5 First citation in articleCrossref, Google Scholar

  • Collaborative on Academic Social Emotional Learning. (2013). Effective social and emotional learning programs: Preschool and elementary school edition. CASEL. First citation in articleGoogle Scholar

  • Collaborative on Academic Social Emotional Learning. (2015). Effective social and emotional learning programs: Middle school and high school. CASEL. First citation in articleGoogle Scholar

  • Collaborative on Academic Social Emotional Learning. (2020, December). SEL competency framework. CASEL. https://casel.org/sel-framework/ First citation in articleGoogle Scholar

  • Denham, S. A. (2015). Assessment of SEL in educational contexts. In J. A. DurlakC. E. DomitrovichR. P. WeissbergT. P. GullottaEds., Handbook of social and emotional learning: Research and practice (pp. 285–300). Guilford Press. First citation in articleGoogle Scholar

  • Doromal, J. B., Cottone, E. A., & Kim, H. (2019). Preliminary validation of the teacher‐rated DESSA in a low‐income, kindergarten sample. Journal of Psychoeducational Assessment, 37(1), 40–54. https://doi.org/10.1177/0734282917731460 First citation in articleCrossref, Google Scholar

  • Durlak, J., Weissberg, R., Dymnicki, A., Taylor, R., & Schellinger, K. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432. https://doi.org/10.1111/j.1467-8624.2010.01564.x First citation in articleCrossref, Google Scholar

  • Elliott, S. N., Anthony, C. J., DiPerna, J. C., Lei, P.-W., & Gresham, F. M. (2020). SSIS SEL Brief Scales: User guide & technical manual. SAIL CoLab. First citation in articleGoogle Scholar

  • Elliott, S. N., DiPerna, J. C., Anthony, C. J., Lei, P. W., & Gresham, F. M. (2020a). SSIS SEL Brief Scales – Teacher K-12. SAIL Collaborative. First citation in articleGoogle Scholar

  • Elliott, S. N., DiPerna, J. C., Anthony, C. J., Lei, P. W., & Gresham, F. M. (2020b). SSIS SEL Brief Scales – Parent K-12. SAIL Collaborative. First citation in articleGoogle Scholar

  • Elliott, S. N., DiPerna, J. C., Anthony, C. J., Lei, P. W., & Gresham, F. M. (2020c). SSIS SEL Brief Scales – Student. SAIL Collaborative. First citation in articleGoogle Scholar

  • Elliott, S. N., Frey, J., & Davies, M. (2015). Systems for assessing and improving students’ social skills to achieve academic competence. In J. DurlakC. DomitrovichR. WeissbergT. Gullotta (Eds.), Handbook for social and emotional learning (pp. 301–319). Guilford Press. First citation in articleGoogle Scholar

  • Elliott, S. N., Lei, P. W., Anthony, C. J., & DiPerna, J. C. (in press). Screening the whole social-emotional child: Expanding a Brief SEL Assessment to include emotional behavior concerns. School Psychology Review. https://doi.org/10.1080/2372966X.2020.1857659 First citation in articleGoogle Scholar

  • Epskamp, S. (2015). semPlot: Unified visualizations of structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 22(3), 474–483. https://doi.org/10.1080/10705511.2014.937847 First citation in articleCrossref, Google Scholar

  • Frye, K. E., Boss, D. L., Anthony, C. J., Du, H., & Xing, W. (2022). Content analysis of the CASEL framework using K-12 State SEL Standards. School Psychology Review. Advance online publication. https://doi.org/10.1080/2372966X.2022.2030193 First citation in articleCrossref, Google Scholar

  • Geske, A., Kampmane, K., & Ozola, A. (2021). The influence of school factors on students’ self-concept: Results from PIRLS 2016. In L. DanielaEd., Human technologies and quality of education (pp. 227–241). https://www.apgads.lu.lv/en/izdevumi/htqe-2021/16/ First citation in articleGoogle Scholar

  • Greenberg, M. T., Weissberg, R. P., O’Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., & Elias, M. J. (2003). Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. American Psychologist, 58(6–7), 466–474. https://doi.org/10.1037/0003-066X.58.6-7.466 First citation in articleCrossref, Google Scholar

  • Gresham, F. M., & Elliott, S. N. (1990). Social skills rating system. AGS. First citation in articleGoogle Scholar

  • Gresham, F. M., & Elliott, S. N. (2008). Social skills improvement system. Pearson. First citation in articleGoogle Scholar

  • Gresham, F. M., & Elliott, S. N. (2017). Social skills improvement system social emotional learning edition rating forms. Pearson Assessments. First citation in articleGoogle Scholar

  • Gresham, F., Elliott, S., Metallo, S., Byrd, S., Wilson, E., Erickson, M., Cassidy, K., & Altman, R. (2020). Psychometric fundamentals of the social skills improvement system: Social–emotional learning edition rating forms. Assessment for Effective Intervention, 45(3), 194–209. https://doi.org/10.1177/1534508418808598 First citation in articleCrossref, Google Scholar

  • Han, K., Colarelli, S. M., & Weed, N. C. (2019). Methodological and statistical advances in the consideration of cultural diversity in assessment: A critical review of group classification and measurement invariance testing. Psychological Assessment, 31(12), 1481–1496. https://doi.org/10.1037/pas0000731 First citation in articleCrossref, Google Scholar

  • Hernandez, A., Hidalgo, M. D., Hambleton, R. K., & Gomez-Benitor, J. (2020). International Test Commission guidelines for test adaptation: A criterion checklist. Psicothema, 32(3), 390–398. First citation in articleGoogle Scholar

  • Hu, T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. https://doi.org/10.1080/10705519909540118 First citation in articleCrossref, Google Scholar

  • Humphrey, N., Lendrum, A., & Wigelsworth, M. (2013). Making the most out of school-based prevention: Lessons from the social and emotional aspects of learning (SEAL) programme. Emotional and Behavioural Difficulties, 18(3), 248–260. https://doi.org/10.1080/13632752.2013.819251 First citation in articleCrossref, Google Scholar

  • International Test Commission. (2017). ITC Guidelines for translating and adapting tests (2nd ed.). https://www.intestcom.org First citation in articleGoogle Scholar

  • Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2018). semTools: Useful tools for structural equation modeling (R package Version 0.5-3) [Computer software]. https://CRAN.R-project.org/package=semTools First citation in articleGoogle Scholar

  • Little, T. D. (2013). Longitudinal structural equation modeling. Guilford Press. First citation in articleGoogle Scholar

  • Liu, Y., Millsap, R. E., West, S. G., & Grimm, K. J. (2017). Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychological Methods, 22(3), 486–506. https://doi.org/10.1037/met0000075 First citation in articleCrossref, Google Scholar

  • Maag, J. (2005). Social skills training for youth with emotional and behavioral disorders and learning disabilities: Problems, conclusions, and suggestions. Exceptionality, 13, 155–172. https://doi.org/10.1207/s15327035ex1303_2 First citation in articleCrossref, Google Scholar

  • OECD. (2015). Skills for social progress: The power of social and emotional skills. OECD Skills Studies, OECD Publishing. First citation in articleCrossref, Google Scholar

  • Panayiotou, M., Santos, J., Black, L., & Humphrey, N. (2022). Exploring the dimensionality of the social skills improvement system using exploratory graph analysis and bifactor-(S − 1) modeling. Assessment, 29(2), 257–271. https://doi.org/10.1177/1073191120971351 First citation in articleCrossref, Google Scholar

  • Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004 First citation in articleCrossref, Google Scholar

  • R Core Team. (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/ First citation in articleGoogle Scholar

  • Reynolds, C. R., & Kamphaus, R. W. (2004). Behavior assessment system for children (2nd ed.). American Guidance Service. First citation in articleGoogle Scholar

  • Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315 First citation in articleCrossref, Google Scholar

  • Ross, K. M., & Tolan, P. (2018). Social and emotional learning in adolescence: Testing the CASEL model in a normative sample. The Journal of Early Adolescence, 38(8), 1170–1199. https://doi.org/10.1177/0272431617725198 First citation in articleCrossref, Google Scholar

  • Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA). Journal of Statistical Software, 48(2), 1–36. First citation in articleCrossref, Google Scholar

  • Salvia, J., Ysseldyke, J. E., & Witmer, S. (2016). Assessment in special and inclusive education (13th ed.). Cengage Learning. First citation in articleGoogle Scholar

  • Sparrow, S. S., Cicchetti, D., & Balla, D. A. (2005). Vineland adaptive behavior scales (2nd ed.). American Guidance Service. First citation in articleGoogle Scholar

  • Steenkamp, J. B. E., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78–90. https://doi.org/10.1086/209528 First citation in articleCrossref, Google Scholar

  • Svetina, D., Rutkowski, L., & Rutkowski, D. (2020). Multiple-group invariance with categorical outcomes using updated guidelines: an illustration using M plus and the lavaan/semtools packages. Structural Equation Modeling: A Multidisciplinary Journal, 27(1), 111–130. https://doi.org/10.1080/10705511.2019.1602776 First citation in articleCrossref, Google Scholar

  • Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Allyn and Bacon. First citation in articleGoogle Scholar

  • Taylor, R. D., Oberle, E., Durlak, J., & Weissberg, R. (2017). Promoting positive youth development through school-based social and emotional learning interventions: A meta-analysis of follow-up effects. Child Development, 88, 1156–1171. https://doi.org/10.1111/cdev.12864 First citation in articleCrossref, Google Scholar

  • Thompson, M. S., & Green, S. B. (2006). Evaluating between-group differences in latent variable means. In G. R. HancockR. O. MuellerEds., A second course in structural equation modeling (pp. 119–169). Wiley. First citation in articleGoogle Scholar

  • Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. https://doi.org/10.1177/109442810031002 First citation in articleCrossref, Google Scholar

  • Walker, H. M.Gresham, F. M. (Eds.). (2014). Handbook of evidence-based practices for emotional and behavioral disorders: Applications in schools. Guilford Press. First citation in articleGoogle Scholar

  • Weissberg, R., & Cascarino, J. (2013). Academic learning + social-emotional learning = national priority. Phi Delta Kappa, 95, 8–13. https://doi.org/10.1177/003172171309500203 First citation in articleCrossref, Google Scholar

  • Weissberg, R., Durlak, J., Domitrovich, C., & Gullotta, T. (2015). Social and emotional learning: Past, present, and future. In J. DurlakC. DomitrovichR. WeissbergT. GullottaEds., Handbook for social and emotional learning (pp. 3–19). Guilford Press. First citation in articleGoogle Scholar

  • World Economic Forum. (Eds.) (2016). New vision for education: Fostering social and emotional learning through technology. https://www3.weforum.org/docs/WEF_New_Vision_for_Education.pdf First citation in articleGoogle Scholar

  • Wu, H., & Estabrook, R. (2016). Identification of confirmatory factor analysis models of different levels of invariance for ordered categorical outcomes. Psychometrika, 81(4), 1014–1045. First citation in articleCrossref, Google Scholar