Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Power of Modified Brown-Forsythe and Mixed-Model Approaches in Split-Plot Designs

Published Online:March 22, 2017https://doi.org/10.1027/1614-2241/a000124

Abstract

Abstract. Low precision of the inferences of data analyzed with univariate or multivariate models of the Analysis of Variance (ANOVA) in repeated-measures design is associated to the absence of normality distribution of data, nonspherical covariance structures and free variation of the variance and covariance, the lack of knowledge of the error structure underlying the data, and the wrong choice of covariance structure from different selectors. In this study, levels of statistical power presented the Modified Brown Forsythe (MBF) and two procedures with the Mixed-Model Approaches (the Akaike’s Criterion, the Correctly Identified Model [CIM]) are compared. The data were analyzed using Monte Carlo simulation method with the statistical package SAS 9.2, a split-plot design, and considering six manipulated variables. The results show that the procedures exhibit high statistical power levels for within and interactional effects, and moderate and low levels for the between-groups effects under the different conditions analyzed. For the latter, only the Modified Brown Forsythe shows high level of power mainly for groups with 30 cases and Unstructured (UN) and Autoregressive Heterogeneity (ARH) matrices. For this reason, we recommend using this procedure since it exhibits higher levels of power for all effects and does not require a matrix type that underlies the structure of the data. Future research needs to be done in order to compare the power with corrected selectors using single-level and multilevel designs for fixed and random effects.

References

  • Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on automatic Control, AC-19, 716–723. doi: 10.1109/TAC.1974.1100705 First citation in articleCrossref, Google Scholar

  • Algina, J. & Keselman, H. J. (1997). Testing repeated measures hypotheses when covariance matrices are heterogeneous: Revisiting the robustness of the Welch-James test. Multivariate Behavioral Research, 32, 255–274. First citation in articleCrossref, Google Scholar

  • Algina, J. & Keselman, H. J. (1998). A power comparison of the Welch-James and Improved General Approximation test in the split-plot design. Journal of the Educational and Behavioral Statistics, 23, 152–159. First citation in articleCrossref, Google Scholar

  • Ato, M., Vallejo, G. & Palmer, A. (2013). The two-way mixed model: A long and winding controversy. Psicothema, 25, 130–136. doi: 10.7335/psicothema2012.15 First citation in articleCrossref, Google Scholar

  • Berkovits, I., Hancock, G. R. & Nevitt, J. (2000). Bootstrap resampling approaches for repeated measure designs: Relative robustness to sphericity and normality violations. Educational and Psychological Measurement, 60, 877–892. First citation in articleCrossref, Google Scholar

  • Cohen, J. (1992). Quantitative methods in Psychology. Psychological Bulletin, 112, 155–159. First citation in articleCrossref, Google Scholar

  • Davidson, M. L. (1972). Univariate versus multivariate test in repeated measures experiments. Psychological Bulletin, 77, 446–452. First citation in articleCrossref, Google Scholar

  • Fernández, P., Livacic-Rojas, P., Vallejo, G. & Tuero-Herrero, E. (2014). Where to look for information when planning scientific research in Psychology: Sources and channels. International Journal of Clinical and Health Psychology, 14, 76–82. First citation in articleCrossref, Google Scholar

  • Fitzmaurice, G. M., Laird, N. M. & Ware, J. H. (2004). Applied longitudinal analysis. Hoboken, NJ: Wiley. First citation in articleGoogle Scholar

  • Fleishman, A. I. (1978). A method for simulating non-normal distributions. Psychometrika, 43, 521–532. First citation in articleCrossref, Google Scholar

  • Greven, S. & Kneib, T. (2010). On the behaviour of marginal and conditional AIC in linear mixed models. Biometrika, 97, 1–17. First citation in articleCrossref, Google Scholar

  • Hamaker, E. L., Van Hattum, P., Kuiper, R. M. & Hoijtink, H. (2011). Model selection based on information criteria in multilevel modeling. In J. J. HoxJ. K. RobertsEds., Handbook of advanced multilevel analysis (pp. 231–255). New York, NY: Taylor & Francis. First citation in articleGoogle Scholar

  • Jennrich, R. I. & Schluchter, M. D. (1986). Unbalanced repeated-measures models with structured covariance matrices. Biometrics, 42, 805–820. doi: 10.2307/2530695 First citation in articleCrossref, Google Scholar

  • Kenward, M. G. & Roger, H. J. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53, 983–997. First citation in articleCrossref, Google Scholar

  • Keselman, H. J., Algina, J. & Kowalchuk, R. K. (2001). The analysis of the repeated measures design: A review. British Journal of Mathematical and Statistical Psychology, 54, 1–20. First citation in articleCrossref, Google Scholar

  • Keselman, H. J., Algina, J., Kowalchuk, R. K. & Wolfinger, R. D. (1998). A comparison of two approaches for selecting covariance structures in the analysis of repeated measurements. Communications in Statistics – Simulation and Computation, 27, 591–604. First citation in articleCrossref, Google Scholar

  • Kowalchuk, R. K., Keselman, H. J., Algina, J. & Wolfinger, R. D. (2004). The analysis of repeated measurements with mixed-model adjusted F test. Educational and Psychological Measurements, 64, 224–242. First citation in articleCrossref, Google Scholar

  • Krishnamoorthy, K. & Yu, J. (2004). Modified Nel and Van der Merwe test for the multivariate Behrens-Fisher problem. Statistics & Probability Letters, 66, 161–169. First citation in articleCrossref, Google Scholar

  • Laird, N. M. & Ware, J. H. (1982). Random-effects models for longitudinal data. Biometrics, 38, 963–974. doi: 10.2307/2529876 First citation in articleCrossref, Google Scholar

  • Liu, S., Rovine, M. J. & Molenaar, P. C. M. (2012). Selecting a linear mixed model for longitudinal data: Repeated measures analysis of variance, covariance pattern model, and growth curve approaches. Psychological Methods, 17, 15–30. doi: 10.1037/a0026971 First citation in articleCrossref, Google Scholar

  • Livacic-Rojas, P., Vallejo, G. & Fernández, P. (2010). Analysis of type I error rates of univariate and multivariate procedures in repeated measures designs. Communications in Statistics – Simulation and Computation, 39, 624–640. doi: 10.1080/03610910903548952 First citation in articleCrossref, Google Scholar

  • Livacic-Rojas, P., Vallejo, G., Fernández, P. & Tuero-Herrero, E. (2013). Covariance structures selection and type I error rates in split plot designs. Methodology. European Journal of Research Methods for the Behavioural and Social Sciences, 9, 129–136. doi: 10.1027/1614-2241/a000058 First citation in articleLink, Google Scholar

  • Lix, L. M. & Lloyd, A. M. (2006, April). A comparison of methods for the analysis of doubly multivariate data. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. First citation in articleGoogle Scholar

  • Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166. First citation in articleCrossref, Google Scholar

  • Nel, D. G. (1997). Tests for equality of parameter matrices in two multivariate linear models. Journal of Multivariate Analysis, 61, 29–37. First citation in articleCrossref, Google Scholar

  • Nel, D. G. & Van der Merwe, C. A. (1986). A solution to the multivariate Behrens-Fisher problem. Communications in Statistics – Theory and Methods, 15, 3719–3735. First citation in articleCrossref, Google Scholar

  • Nunez, J. C., Rosario, P., Vallejo, G. & González-Pienda, J. A. (2013). A longitudinal assessment of the effectiveness of a school-based mentoring program in middle school. Contemporary Educational Psychology, 38, 11–21. First citation in articleCrossref, Google Scholar

  • Ramsey, P. H. (1978). Power differences between pairwise multiple comparisons. Journal of the American Statistical Association, 73, 479–485. First citation in articleCrossref, Google Scholar

  • SAS Institute. (2005). The MIXED procedure 2005, SAS/STAT user’s guide, Version 9. SAS On-Line Documentation. Cary, NC: SAS Institute. First citation in articleGoogle Scholar

  • Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464. First citation in articleCrossref, Google Scholar

  • Spiegelhalter, D. J., Best, N. G., Carlin, B. P. & Van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society Series B, 64, 583–640. First citation in articleCrossref, Google Scholar

  • Srivastava, M. S. & Kubokawa, T. (2010). Conditional information criteria for selecting variables in linear mixed models. Journal of Multivariate Analysis, 101, 1970–1980. First citation in articleCrossref, Google Scholar

  • Stroup, W. W. (2002). Power analysis based on spatial effects mixed models: A tool for comparing design and analysis strategies in the presence of spatial variability. Journal of Agricultural, Biological, and Environmental Statistics, 7, 491–511. First citation in articleCrossref, Google Scholar

  • Stroup, W. W. (2013). Generalized linear mixed models. Modern concepts, methods and applications. New York, NY: CRC Press. First citation in articleGoogle Scholar

  • Timm, N. H. (2002). Applied multivariate analysis. New York, NY: Springer. First citation in articleGoogle Scholar

  • Vaida, F. & Blanchard, S. (2005). Conditional Akaike information for mixed-effects models. Biometrika, 92, 351–370. First citation in articleCrossref, Google Scholar

  • Vale, C. D. & Maurelli, V. A. (1983). Simulating multivariate nonnormal distributions. Psychometrika, 48, 465–471. First citation in articleCrossref, Google Scholar

  • Vallejo, G., Arnau, J. & Ato, M. (2007). Comparative robustness of recent methods for analyzing multivariate repeated measures. Educational & Psychological Measurement, 67, 1–27. First citation in articleGoogle Scholar

  • Vallejo, G., Arnau, J., Bono, R., Fernández, M. P. & Tuero-Herrero, E. (2010). Nested model selection for longitudinal data using information criteria and the conditional adjustment strategy. Psicothema, 22, 323–333. First citation in articleGoogle Scholar

  • Vallejo, G. & Ato, M. (2006). Modified Brown-Forsythe procedure for testing interaction effects in split-plot designs. Multivariate Behavioral Research, 41, 549–578. First citation in articleCrossref, Google Scholar

  • Vallejo, G., Ato, M., Fernández, P. & Livacic-Rojas, P. (2013). Multilevel bootstrap analysis with assumptions violated. Psicothema, 25, 520–528. doi: 10.7334/psicothema2013.58 First citation in articleCrossref, Google Scholar

  • Vallejo, G., Ato, M. & Valdés, T. (2008). Consequences of misspecifying the error covariance structure in linear mixed models for longitudinal data. Methodology, 4, 10–21. First citation in articleLink, Google Scholar

  • Vallejo, G., Fernández, M. P., Herrero, J. & Livacic-Rojas, P. (2007). Examen comparativo de la sensibilidad de dos enfoques robustos para detectar los efectos de un diseño doblemente multivariado. Revista mexicana de Psicología, 24, 53–64. First citation in articleGoogle Scholar

  • Vallejo, G., Fernández, M. P. & Livacic-Rojas, P. (2007). Power differences between the modified Brown-Forsythe and mixed-model approaches in repeated measures designs. Methodology, 3, 1–13. First citation in articleLink, Google Scholar

  • Vallejo, G., Fernández, M. P., Livacic-Rojas, P. E. & Tuero-Herrero, E. (2011a). Selecting the best unbalanced repeated measures model. Behavior Research Methods, 43, 18–36. First citation in articleCrossref, Google Scholar

  • Vallejo, G., Fernández, P., Livacic-Rojas, P. & Tuero-Herrero, E. (2011b). Comparison of modern methods for analyzing repeated measures data with missing values. Multivariate Behavioral Research, 46, 900–937. First citation in articleCrossref, Google Scholar

  • Vallejo, G., Fidalgo, A. M. & Fernández, P. (2001). Effects of covariance heterogeneity on three procedures for analysing multivariate repeated measures designs. Multivariate Behavioral Research, 36, 1–27. First citation in articleCrossref, Google Scholar

  • Vallejo, G. & Livacic-Rojas, P. (2005). A comparison of two procedures for analyzing small sets of repeated measures data. Multivariate Behavioral Research, 40, 179–205. First citation in articleCrossref, Google Scholar

  • Vallejo, G., Tuero-Herrero, E., Núñez, J. C. & Rosário, P. (2014). Performance evaluation of recent information criteria for selecting multilevel models in Behavioral and Social Sciences. International Journal of Clinical and Health Psychology, 14, 48–57. First citation in articleCrossref, Google Scholar

  • Wilcox, R. (2012). Introduction to Robust Estimation & Hypothesis Testing (3rd ed.). Waltham, UK: Elsevier. First citation in articleGoogle Scholar