Model Error in Covariance Structure Models
Some Implications for Power and Type I Error
Abstract
This study investigated the degree to which violation of the parameter drift assumption affects the Type I error rate for the test of close fit and the power analysis procedures proposed by MacCallum et al. (1996) for both the test of close fit and the test of exact fit. The parameter drift assumption states that as sample size increases both sampling error and model error (i.e., the degree to which the model is an approximation in the population) decrease. Model error was introduced using a procedure proposed by Cudeck and Browne (1992). The empirical power for both the test of close fit, in which the null hypothesis specifies that the root mean square error of approximation (RMSEA) ≤ 0.05, and the test of exact fit, in which the null hypothesis specifies that RMSEA = 0, is compared with the theoretical power computed using the MacCallum et al. (1996) procedure. The empirical power and the theoretical power for both the test of close fit and the test of exact fit are nearly identical under violations of the assumption. The results also indicated that the test of close fit maintains the nominal Type I error rate under violations of the assumption.
References
(1982). Covariance structures. In , Topics in Applied Multivariate Analysis. New York: Cambridge University Press.
(1993). Alternative ways of assessing model fit. In , Testing Structural Equation Models. Newbury Park, CA: Sage.
(1992). Constructing a covariance matrix that yields a specified minimizer and a specified minimum discrepancy function value. Psychometrika, 57(3), 357–369.
(1991). Model selection in covariance structures analysis and the “problem” of sample size: A clarification. Psychological Bulletin, 109, 512–519.
(2003). A realistic perspective on pattern representation in growth data: Comment on Bauer and Curran. Psychological Methods, 8, 378–383.
(1995). Statistical power in structural equation modeling. In , Structural Equation Modeling: Concepts, Issues, and Applications. Thousand Oaks, CA: Sage.
(1999). Elements of Large Sample Theory. New York: Springer.
(2003). Working with imperfect models. Multivariate Behavioral Research, 38(1), 113–139.
(1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130–149.
(1997). Power analysis in covariance structure modeling using GFI and AGFI. Multivariate Behavioral Research, 32(2), 193–210.
(1991). Representing sources of error in the common-factor model: Implications for theory and practice. Psychological Bulletin, 109(3), 502–511.
(2006). Random intercept item factor analysis. Psychological Methods, 11, 344–362.
(2003). Marketing orientation and business economic performance. International Journal of Service Industry Management, 14, 284–309.
(2001). Numerical Methods of Statistics. New York: Cambridge University Press.
(2006, May). Computing power and minimum sample size for RMSEA [computer software], Available from www.quantpsy.org.
(1993). Power evaluations in structural equation models. In , Testing Structural Equation Models. Newbury Park, CA: Sage.
(1987). The detection and correction of specification errors in structural equation models. Sociological Methodology, 17, 105–129.
(1985). Power of the likelihood ratio test in covariance structure analysis. Psychometrika, 50(1), 83–90.
(1980, June). Statistically based tests for the number of common factors. Paper presented at the annual meeting of the Psychometric Society, Iowa City, IA.
(1985). On the multivariate asymptotic distribution of sequential chi-square statistics. Psychometrika, 50(3), 253–264.
