Power Differences Between the Modified Brown-Forsythe and Mixed-Model Approaches in Repeated Measures Designs
Abstract
Abstract. This article compares the sensibility of the modified Brown-Forsythe (MBF) approach developed by Vallejo and Ato (2006) and a modified empirical generalized least squares (EGLS) method adjusted by the Kenward-Roger solution available in the SAS Institute's (2002)Proc Mixed program to detect the presence of an interaction effect under departures from covariance homogeneity and multivariate normality. Although none of the approaches demonstrated superior performance in all situations, our results indicate that the so-called EGLS method, based on the Akaike's Information Criterion or based on always assuming a unstructured between-subjects heterogeneous covariance pattern, was the most powerful alternative. Results also indicate that little power can be achieved with the EGLS method if the covariance matrix is specified correctly.
References
Algina J. , Keselman H.J. (1997). Testing repeated measures hypotheses when covariance matrices are heterogeneous: Revisiting the robustness of the Welch-James test. Multivariate Behavioral Research, 32, 255– 274Algina J. , Keselman H.J. (1998). A power comparison of the Welch-James and Improved General Approximation tests in the split-plot design. Journal of Educational and Behavioral Statistics, 23, 152– 169Algina J. , Oshima T.C. (1994). Type I error rates for Huynh's General Approximation and Improved General Approximation tests. British Journal of Mathematical and Statistical Psychology, 47, 151– 165Algina J. , Oshima T.C. (1995). An improved general approximation test for the main effect in a split-plot design. British Journal of Mathematical and Statistical Psychology, 48, 149– 160Box G.E.P. (1954). Some theorems on quadratic forms applied in the study of analysis of variance problems. I. Effects of inequality of variance in the one-way classification. Annals of Mathematical Statistics, 25, 290– 403Brown M.B. , Forsythe A.B. (1974). The small-sample behavior of some statistics which test the equality of several means. Technometrics, 16, 129– 132Chen X. , Wei L. (2003). A comparison of recent methods for the analysis of small-sample cross-over studies. Statistics in Medicine, 22, 2821– 2833Cohen J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: LEAFitzmaurice G.M. , Laird N.M. , Ware J.H. (2004). Applied longitudinal analysis . Hoboken, NJ: WileyFleishman A.I. (1978). A method for simulating nonnormal distributions. Psychometrika, 43, 521– 532Huynh H. (1978). Some approximate tests for repeated measurement designs. Psychometrika, 43, 161– 165Jennrich R.I. , Schluchter M.D. (1986). Unbalanced repeated-measures models with structured covariance matrices. Biometrics, 42, 805– 820Johansen S. (1980). The Welch-James approximation to the distribution of the residual sum of squares in a weighted linear regression. Biometrika, 67, 85– 92Kenward M.G. (2001). Comments on “Parametric modelling of growth curve data: An overview,” by D.L. Zimmerman & V. Núñez-Antón. Test, 10, 1– 73Kenward M.G. , Roger J.H. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53, 983– 997Keselman H.J. , Algina J. , Wilcox R.K. , Kowalchuk R.K. (2000). Testing repeated measures hypotheses when covariance matrices are heterogeneous: Revisiting the robustness of the Welch-James test again. Educational and Psychological Measurement, 60, 925– 938Kowalchuk R.K. , Keselman H.J. , Algina J. , Wolfinger R.D. (2004). The analysis of repeated measurements with mixed-model adjusted F tests. Educational and Psychological Measurement, 64, 224– 242Krishnamoorthy K. , Yu J. (2004). Modified Nel and Van der Merwe test for the multivariate Behrens-Fisher problem. Statistics & Probability Letters, 66, 161– 169Laird N.M. , Ware J.H. (1982). Random-effects models for longitudinal data. Biometrics, 38, 963– 974Lix L.M. , Algina J. , Keselman H.J. (2003). Analysing multivariate repeated measures designs: A comparison of two approximate degrees of freedom procedures. Multivariate Behavioral Research, 38, 403– 431Lix L.M. , Lloyd A.M. (2006, April). A comparison of methods for the analysis of doubly multivariate data . Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CAMicceri T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 92, 778– 785Muller K.E. , La Vange L.M. , Ramey S.H. , Ramey C.T. (1992). Power calculations for general lineal multivariate models including repeated measures applications. Journal of the American Statistical Association, 87, 1209– 1226Nel D.G. (1997). Tests for equality of parameter matrices in two multivariate linear models. Journal of Multivariate Analysis, 61, 29– 37Nel D.G. , van der Merwe C.A. (1986). A solution to the multivariate Behrens-Fisher problem. Communications in Statistics - Theory and Methods, 15, 3719– 3735Olson C.L. (1974). Comparative robustness of six tests in multivariate analysis of variance. Journal of the American Statistical Association, 69, 894– 908Raudenbush S.W. , Bryk A.S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed,). Thousand Oaks, CA: Sage(2002). SAS/STAT software: Version 9,0 . Cary, NC: Author
Stroup W.W. (2002). Power analysis based on spatial effects mixed models: A tool for comparing design and analysis strategies in the presence of spatial variability. Journal of Agricultural, Biological, and Environmental Statistics, 7, 491– 511Timm N.H. (2002). Applied multivariate analysis . New York: SpringerVale C.D. , Maurelli V.A. (1983). Simulating multivariate nonnormal distributions. Psychometrika, 48, 465– 471Vallejo G. , Ato M. (2006). Modified Brown-Forsythe procedure for testing interaction effects in split-plot designs. Multivariate Behavioral Research, 41, 549– 578Vallejo G. , Cuesta M. , Fernández P. , Herrero J. (2006). A comparison of the bootstrap-F, improved general approximation, and Brown-Forsythe multivariate approaches in a mixed repeated measures design. Educational and Psychological Measurement, 66, 35– 62Vallejo G. , Livacic-Rojas P. (2005). A comparison of two procedures for analyzing small sets of repeated measures data. Multivariate Behavioral Research, 40, 179– 205Vallejo G. , Moris J. , Conejo N.M. (2006). A SAS/IML program for implementing the modified Brown-Forsythe procedure in repeated measures designs. Computer Methods and Programs in Biomedicine, 83, 169– 177Zimmerman D.L. , Núñez-Antón V. (2001). Parametric modeling of growth curve data: An overview. Test, 10, 1– 73
