Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Original Article

Pushing the Limits

The Performance of Maximum Likelihood and Bayesian Estimation With Small and Unbalanced Samples in a Latent Growth Model

Published Online:December 12, 2018https://doi.org/10.1027/1614-2241/a000162

Abstract

Abstract. Longitudinal developmental research is often focused on patterns of change or growth across different (sub)groups of individuals. Particular to some research contexts, developmental inquiries may involve one or more (sub)groups that are small in nature and therefore difficult to properly capture through statistical analysis. The current study explores the lower-bound limits of subsample sizes in a multiple group latent growth modeling by means of a simulation study. We particularly focus on how the maximum likelihood (ML) and Bayesian estimation approaches differ when (sub)sample sizes are small. The results show that Bayesian estimation resolves computational issues that occur with ML estimation and that the addition of prior information can be the key to detect a difference between groups when sample and effect sizes are expected to be limited. The acquisition of prior information with respect to the smaller group is especially influential in this context.

References

  • Asparouhov, T. & Muthén, B. O. (2010, September). Bayesian analysis of latent variable models using Mplus. Retrieved from https://www.statmodel.com/techappen.shtml First citation in articleGoogle Scholar

  • Boomsma, A. & Hoogland, J. J. (2001). The robustness of LISREL modeling revisited. In R. CudeckK. G. JöreskogD. SörbomEds., Structural equation models: Present and future. A festschrift in honor of Karl Jöreskog (pp. 139–168). Lincolnwood, IL: Scientific Software International. First citation in articleGoogle Scholar

  • Can, S., van de Schoot, R. & Hox, J. (2015). Collinear latent variables in multilevel confirmatory factor analysis a comparison of maximum likelihood and Bayesian estimations. Educational and Psychological Measurement, 75, 406–427. https://doi.org/10.1177/0013164414547959 First citation in articleCrossref, Google Scholar

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. https://doi.org/10.4324/9780203771587 First citation in articleGoogle Scholar

  • Depaoli, S. (2013). Mixture class recovery in GMM under varying degrees of class separation: Frequentist versus Bayesian estimation. Psychological Methods, 18, 186. https://doi.org/10.1037/a0031609 First citation in articleCrossref, Google Scholar

  • Depaoli, S. & van de Schoot, R. (2015). Improving transparency and replication in Bayesian statistics: The WAMBS-checklist. Psychological Methods, 22(2), 240–261. https://doi.org/10.1037/met0000065 First citation in articleCrossref, Google Scholar

  • Gelman, A. & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472. https://doi.org/10.1214/ss/1177011136 First citation in articleCrossref, Google Scholar

  • Hallquist, M. (2013, October). MplusAutomation: Automating Mplus model estimation and interpretation. Package MplusAutomation. Retrieved from https://cran.r-project.org/web/packages/MplusAutomation/MplusAutomation.pdf First citation in articleGoogle Scholar

  • Hochweber, J. & Hartig, J. (2017). Analyzing organizational growth in repeated cross-sectional designs using multilevel structural equation modeling. Methodology, 13, 83–97. https://doi.org/10.1027/1614-2241/a000133 First citation in articleLink, Google Scholar

  • Hox, J. & Maas, C. J. (2001). The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Structural Equation Modeling, 8, 157–174. https://doi.org/10.1207/S15328007SEM08021 First citation in articleCrossref, Google Scholar

  • Hox, J., Moerbeek, M., Kluytmans, A. & van de Schoot, R. (2014). Analyzing indirect effects in cluster randomized trials. The effect of estimation method, number of groups and group sizes on accuracy and power. Frontiers in Psychology, 5, 78. https://doi.org/10.3389/fpsyg.2014.00078 First citation in articleCrossref, Google Scholar

  • Hox, J., van de Schoot, R. & Matthijse, S. (2012). How few countries will do? Comparative survey analysis from a Bayesian perspective. Survey Research Association, 6, 87–93. https://doi.org/10.18148/srm/2012.v6i2.5033 First citation in articleGoogle Scholar

  • Jacobus, J., Bava, S., Cohen-Zion, M., Mahmood, O. & Tapert, S. (2009). Functional consequences of marijuana use in adolescents. Pharmacology, Biochemistry and Behavior, 4, 559–565. https://doi.org/10.1016/j.pbb.2009.04.001 First citation in articleCrossref, Google Scholar

  • Kruschke, J. K. (2011). Introduction to special section on Bayesian data analysis. Perspectives on Psychological Science, 6, 272–273. https://doi.org/10.1177/1745691611406926 First citation in articleCrossref, Google Scholar

  • Kruschke, J. K. (2014). Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan (2nd ed.). San Diego, CA: Academic Press. First citation in articleGoogle Scholar

  • Lee, S.-Y. & Song, X.-Y. (2004). Evaluation of the Bayesian and maximum likelihood approaches in analyzing structural equation models with small sample sizes. Multivariate Behavioral Research, 39, 653–686. https://doi.org/10.1207/s15327906mbr3904_4 First citation in articleCrossref, Google Scholar

  • Little, T. D. (2013). Longitudinal structural equation modeling. New York, NY: Guilford Press. First citation in articleGoogle Scholar

  • Lüdtke, O., Marsh, H. W., Robitzsch, A. & Trautwein, U. (2011). A 2 × 2 taxonomy of multilevel latent contextual models: Accuracy-bias trade-offs in full and partial error correction models. Psychological Methods, 16, 444. https://doi.org/10.1037/a0024376 First citation in articleCrossref, Google Scholar

  • Maas, C. J. & Hox, J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1, 86–92. https://doi.org/10.1027/1614-1881.1.3.86 First citation in articleLink, Google Scholar

  • McNeish, D. M. (2016a). On using Bayesian methods to address small sample problems. Structural Equation Modeling, 23, 750–773. https://doi.org/10.1080/10705511.2016.1186549 First citation in articleCrossref, Google Scholar

  • McNeish, D. M. (2016b). Using data-dependent priors to mitigate small sample bias in latent growth models: A discussion and illustration using Mplus. Journal of Educational and Behavioral Statistics, 41, 27–56. https://doi.org/10.3102/1076998615621299 First citation in articleCrossref, Google Scholar

  • Meuleman, B. & Billiet, J. (2009). A Monte Carlo sample size study: How many countries are needed for accurate multilevel SEM? Survey Research Methods, 3, 45–58. https://doi.org/10.18148/srm/2009.v3i1.666 First citation in articleGoogle Scholar

  • Muthén, B. O. & Curran, P. J. (1997). General longitudinal modeling of individual differences in experimental designs: A latent variable framework for analysis and power estimation. Psychological Methods, 2, 371–402. https://doi.org/10.1037/1082-989X.2.4.371 First citation in articleCrossref, Google Scholar

  • Muthén, L. K. & Muthén, B. O. (1998–2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén. First citation in articleGoogle Scholar

  • Muthén, L. K. & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9, 599–620. https://doi.org/10.1207/S15328007SEM0904_8 First citation in articleCrossref, Google Scholar

  • Peeters, M., Monshouwer, K., Janssen, T., Wiers, R. W. & Vollebergh, W. A. (2014). Working memory and alcohol use in at-risk adolescents: A 2-year follow-up. Alcoholism: Clinical and Experimental Research, 38, 1176–1183. https://doi.org/10.1111/acer.12339 First citation in articleCrossref, Google Scholar

  • R Core Team. (2015). R: A language and environment for statistical computing [Computer software manual]. Vienna, Austria. Retrieved from https://www.R-project.org/ First citation in articleGoogle Scholar

  • Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1–36. Retrieved from https://www.jstatsoft.org/article/view/v048i02 First citation in articleCrossref, Google Scholar

  • Tolvanen, A. (2000). Latenttien kasvukäyrä- ja simplex-mallien teoriaa ja sovelluksia pitkittäisaineistoissa kehityksen ja muutoksen analysointiin [Latent growth and simplex models: Theory and applications in longitudinal models for analysis of development and change]. Jyväskylä, Finland: Department of Statistics, University of Jyväskylä. First citation in articleGoogle Scholar

  • van de Schoot, R., Broere, J. J., Perryck, K. H., Zondervan-Zwijnenburg, M. & Van Loey, N. E. (2015). Analyzing small data sets using Bayesian estimation: The case of posttraumatic stress symptoms following mechanical ventilation in burn survivors. European Journal of Psychotraumatology, 6, 25216. https://doi.org/10.3402/ejpt.v6.25216 First citation in articleCrossref, Google Scholar

  • van de Schoot, R., Kaplan, D., Denissen, J., Asendorpf, J. B., Neyer, F. J. & Van Aken, M. A. (2013). A gentle introduction to Bayesian analysis: Applications to developmental research. Child Development, 85, 842–860. https://doi.org/10.1111/cdev.12169 First citation in articleCrossref, Google Scholar

  • van de Schoot, R., Winter, S. D., Ryan, O., Zondervan-Zwijnenburg, M. & Depaoli, S. (2017). A systematic review of Bayesian articles in psychology: The last 25 years. Psychological Methods, 22, 217–239. https://doi.org/10.1037/met0000100 First citation in articleCrossref, Google Scholar

  • Zondervan-Zwijnenburg, M., Peeters, M., Depaoli, S. & van de Schoot, R. (2017). Where do priors come from? Applying guidelines to construct informative priors in small sample research. Research in Human Development, 14, 305–320. https://doi.org/10.1080/15427609.2017.1370966 First citation in articleCrossref, Google Scholar