Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Next article
Article

Analyzing Organizational Growth in Repeated Cross-Sectional Designs Using Multilevel Structural Equation Modeling

Published Online:September 08, 2017https://doi.org/10.1027/1614-2241/a000133

Abstract

Abstract. In repeated cross-sections of organizations, different individuals are sampled from the same set of organizations at each time point of measurement. As a result, common longitudinal data analysis methods (e.g., latent growth curve models) cannot be applied in the usual way. In this contribution, a multilevel structural equation modeling approach to analyze data from repeated cross-sections is presented. Results from a simulation study are reported which aimed at obtaining guidelines on appropriate sample sizes. We focused on a situation where linear growth occurs at the organizational level, and organizational growth is predicted by a single organizational level variable. The power to identify an effect of this organizational level variable was moderately to strongly positively related to number of measurement occasions, number of groups, group size, intraclass correlation, effect size, and growth curve reliability. The Type I error rate was close to the nominal alpha level under all conditions.

References

  • Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In S. W. J. KozlowskiK. J. KleinEds., Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 349–381). San Francisco, CA: Jossey-Bass. First citation in articleGoogle Scholar

  • Bollen, K. A., & Curran, P. J. (2006). Latent curve models: A structural equation perspective. Hoboken, NJ: Wiley. First citation in articleGoogle Scholar

  • Bradley, J. V. (1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31, 144–152. doi: 10.1111/j.2044-8317.1978.tb00581.x First citation in articleCrossref, Google Scholar

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Creemers, B. P. M., & Kyriakides, L. (2010). Explaining stability and changes in school effectiveness by looking at changes in the functioning of school factors. School Effectiveness and School Improvement, 21, 409–427. doi: 10.1080/09243453.2010.512795 First citation in articleCrossref, Google Scholar

  • Deaton, A. (1985). Panel data from time series of cross-sections. Journal of Econometrics, 30, 109–126. doi: 10.1016/0304-4076(85)90134-4 First citation in articleCrossref, Google Scholar

  • Donner, A., & Klar, N. (2000). Design and analysis of cluster randomization trials in health research. London, UK: Arnold. First citation in articleGoogle Scholar

  • Duncan, T. E., Duncan, S. C., & Li, F. (1998). A comparison of model-and multiple imputation-based approaches to longitudinal analyses with partial missingness. Structural Equation Modeling, 5, 1–21. doi: 10.1080/10705519809540086 First citation in articleCrossref, Google Scholar

  • Fan, X. (2003). Power of latent growth modeling for detecting group differences in linear growth trajectory parameters. Structural Equation Modeling, 10, 380–400. doi: 10.1207/S15328007SEM1003_3 First citation in articleCrossref, Google Scholar

  • Fan, X., & Fan, X. (2005). Power of latent growth modeling for detecting linear growth: Number of measurements and comparison with other analytic approaches. The Journal of Experimental Education, 73, 121–139. doi: 10.3200/JEXE.73.2.121-139 First citation in articleCrossref, Google Scholar

  • Feldman, H. A., & McKinlay, S. M. (1994). Cohort versus cross-sectional design in large field trials: Precision, sample size, and a unifying model. Statistics in Medicine, 13, 61–78. doi: 10.1002/sim.4780130108 First citation in articleCrossref, Google Scholar

  • Ganzeboom, H. B. G., de Graaf, P. M., Treiman, D. J., & de Leeuw, J. (1992). A standard international socio-economic index of occupational status. Social Science Research, 21, 1–56. doi: 10.1016/0049-089X(92)90017-B First citation in articleCrossref, Google Scholar

  • Genz, A., Bretz, F., Miwa, T., Mi, X., Leisch, F., Scheipl, F., & Hothorn, T. (2014). mvtnorm: Multivariate Normal and t Distributions. R package version 1.0-0. Retrieved from http://CRAN.R-project.org/package=mvtnorm First citation in articleGoogle Scholar

  • Gray, J., Jesson, D., Goldstein, H., Hedger, K., & Rasbash, J. (1995). A multi-level analysis of school improvement: Changes in schools’ performance over time. School Effectiveness and School Improvement, 6, 97–114. doi: 10.1080/0924345950060201 First citation in articleCrossref, Google Scholar

  • Greenwald, R., Hedges, L. V., & Laine, R. D. (1996). The effect of school resources on student achievement. Review of Educational Research, 66, 361–396. doi: 10.3102/00346543066003361 First citation in articleCrossref, Google Scholar

  • Hedges, L. V., & Hedberg, E. C. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29, 60–87. doi: 10.3102/0162373707299706 First citation in articleCrossref, Google Scholar

  • Hertzog, C., Lindenberger, U., Ghisletta, P., & Von Oertzen, T. (2006). On the power of multivariate latent growth curve models to detect correlated change. Psychological Methods, 11, 244–252. doi: 10.1037/1082-989X.11.3.244 First citation in articleCrossref, Google Scholar

  • Hertzog, C., Von Oertzen, T., Ghisletta, P., & Lindenberger, U. (2008). Evaluating the power of latent growth curve models to detect individual differences in change. Structural Equation Modeling, 15, 541–563. doi: 10.1080/10705510802338983 First citation in articleCrossref, Google Scholar

  • Hooper, R., & Bourke, L. (2015). Cluster randomised trials with repeated cross sections: Alternatives to parallel group designs. British Medical Journal, 350, h2925. doi: 10.1136/bmj.h2925 First citation in articleCrossref, Google Scholar

  • Hox, J. J., & Maas, C. J. M. (2001). The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Structural Equation Modeling, 8, 157–174. doi: 10.1207/S15328007SEM0802_1 First citation in articleCrossref, Google Scholar

  • Hox, J. J., Maas, C. J. M., & Brinkhuis, M. J. S. (2010). The effect of estimation method and sample size in multilevel structural equation modeling. Statistica Neerlandica, 64, 157–170. doi: 10.1111/j.1467-9574.2009.00445.x First citation in articleCrossref, Google Scholar

  • Kaplan, D. (1995). Statistical power in structural equation modeling. In R. H. HoyleEd., Structural equation modeling. Concepts, issues, and applications (pp. 100–117). Thousand Oaks, CA: Sage. First citation in articleGoogle Scholar

  • Kaplan, D., & Elliott, P. R. (1997). A didactic example of multilevel structural equation modeling applicable to the study of organizations. Structural Equation Modeling, 4, 1–24. doi: 10.1080/10705519709540056 First citation in articleCrossref, Google Scholar

  • Konstantopoulos, S. (2010). Power analysis in two-level unbalanced designs. Journal of Experimental Education, 78, 291–317. doi: 10.1080/00220970903292876 First citation in articleCrossref, Google Scholar

  • LaHuis, D. M., & Ferguson, M. W. (2009). The accuracy of significance tests for slope variance components in multilevel random coefficient models. Organizational Research Methods, 12, 418–435. doi: 10.1177/1094428107308984 First citation in articleCrossref, Google Scholar

  • Li, X., & Beretvas, S. N. (2013). Sample size limits for estimating upper level mediation models using multilevel SEM. Structural Equation Modeling, 20, 241–264. doi: 10.1080/10705511.2013.769391 First citation in articleCrossref, Google Scholar

  • Little, T. D. (2013). Longitudinal structural equation modeling. New York, NY: The Guilford Press. First citation in articleGoogle Scholar

  • Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. O. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13, 203–229. doi: 10.1037/a0012869 First citation in articleCrossref, Google Scholar

  • Ma, X., & Willms, J. D. (2004). School disciplinary climate: Characteristics and effects on eighth grade achievement. Alberta Journal of Educational Research, 50, 169–188. First citation in articleGoogle Scholar

  • Maas, C., & Hox, J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1, 85–91. doi: 10.1027/1614-2241.1.3.86 First citation in articleLink, Google Scholar

  • McArdle, J. J. (2009). Latent variable modeling of differences and changes with longitudinal data. Annual Review of Psychology, 60, 577–605. doi: 10.1146/annurev.psych.60.110707.163612 First citation in articleCrossref, Google Scholar

  • McArdle, J. J., & Nesselroade, J. R. (2014). Longitudinal data analysis using structural equation models. Washington, DC: American Psychological Association. First citation in articleCrossref, Google Scholar

  • McNeish, D. M., & Stapleton, L. M. (2016). The effect of small sample size on two-level model estimates: A review and illustration. Educational Psychology Review, 28, 295–314. doi: 10.1007/s10648-014-9287-x First citation in articleCrossref, Google Scholar

  • Mehta, P. D., & Neale, M. C. (2005). People are variables too: Multilevel structural equations modeling. Psychological Methods, 10, 259–284. doi: 10.1037/1082-989X.10.3.259 First citation in articleCrossref, Google Scholar

  • Meredith, W., & Tisak, J. (1990). Latent curve analysis. Psychometrika, 55, 107–122. doi: 10.1007/BF02294746 First citation in articleCrossref, Google Scholar

  • Muthén, B. O. (1991). Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement, 28, 338–354. doi: 10.1111/j.1745-3984.1991.tb00363.x First citation in articleCrossref, Google Scholar

  • Muthén, B. O., & Asparouhov, T. (2011). Beyond multilevel regression modeling: Multilevel analysis in a general latent variable framework. In J. J. HoxJ. K. RobertsEds., Handbook of advanced multilevel analysis (pp. 15–40). New York, NY: Routledge. First citation in articleGoogle Scholar

  • Muthén, B. O., & Curran, P. J. (1997). General longitudinal modeling of individual differences in experimental designs: A latent variable framework for analysis and power estimation. Psychological Methods, 2, 371–402. doi: 10.1037/1082-989X.2.4.371 First citation in articleCrossref, Google Scholar

  • Muthén, L. K., & Muthén, B. O. (1998-2010). Mplus – Statistical analysis with latent variables [Computer software]. Muthén & Muthén: Los Angeles, CA. First citation in articleGoogle Scholar

  • Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9, 599–620. doi: 10.1207/S15328007SEM0904_8 First citation in articleCrossref, Google Scholar

  • Organisation for Economic Cooperation and Development. (2009). PISA 2006 technical report. Paris, France: OECD. First citation in articleGoogle Scholar

  • Preacher, K. J. (2015). Advances in mediation analysis: A survey and synthesis of new developments. Annual Review of Psychology, 66, 825–852. doi: 0.1146/annurev-psych-010814-015258 First citation in articleCrossref, Google Scholar

  • Preacher, K. J., Wichman, A. L., MacCallum, R. C., & Briggs, N. E. (2008). Latent growth curve modeling. Thousand Oaks, CA: Sage. First citation in articleCrossref, Google Scholar

  • R Core Team. (2013). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing: Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org/ First citation in articleGoogle Scholar

  • Ryu, E. (2014). Model fit evaluation in multilevel structural equation models. Frontiers in Psychology, 5, 81. doi: 10.3389/fpsyg.2014.00081 First citation in articleCrossref, Google Scholar

  • Shevlin, M., & Millar, R. (2006). Career education: An application of latent growth curve modelling to career information-seeking behaviour of school pupils. British Journal of Educational Psychology, 76, 141–153. doi: 10.1348/000709904x22386 First citation in articleCrossref, Google Scholar

  • Simons-Morton, B., Chen, R., Abroms, L., & Haynie, D. L. (2004). Latent growth curve analyses of peer and parent influences on smoking progression among early adolescents. Health Psychology, 23, 612–621. doi: 10.1037/0278-6133.23.6.612 First citation in articleCrossref, Google Scholar

  • Sprietsma, M. (2012). Computers as pedagogical tools in Brazil: A pseudo-panel analysis. Education Economics, 20, 19–32. doi: 10.1080/09645290903546496 First citation in articleCrossref, Google Scholar

  • Thomas, S., Sammons, P., Mortimore, P., & Smees, R. (1997). Stability and consistency in secondary schools’ effects on students’ GCSE outcomes over three years. School Effectiveness and School Improvement, 8, 169–197. doi: 10.1080/0924345970080201 First citation in articleCrossref, Google Scholar

  • Ukoumunne, O. C., & Thompson, S. G. (2001). Analysis of cluster randomized trials with repeated cross-sectional binary measurements. Statistics in Medicine, 20, 417–433. doi: 10.1002/1097-0258(20010215)20:3<417::AID-SIM802>3.0.CO;2-G First citation in articleCrossref, Google Scholar

  • Verbeek, M. (2008). Pseudo-panels and repeated cross-sections. In L. MátyásP. SevestreEds., The econometrics of panel data (pp. 369–383). Berlin, Germany: Springer. First citation in articleGoogle Scholar

  • Verbeek, M., & Nijman, T. (1993). Minimum MSE estimation of a regression model with fixed effects from a series of cross-sections. Journal of Econometrics, 59, 125–136. doi: 10.1016/0304-4076(93)90042-4 First citation in articleCrossref, Google Scholar

  • Willms, J. D., & Raudenbush, S. W. (1989). A longitudinal hierarchical linear model for estimating school effects and their stability. Journal of Educational Measurement, 26, 209–232. doi: 10.1111/j.1745-3984.1989.tb00329.x First citation in articleCrossref, Google Scholar