Abstract
Abstract.Culpepper and Aguinis (2011) highlighted the benefit of using the errors-in-variables (EIV) method to control for measurement error and obtain unbiased regression estimates. The current study investigated the EIV method and compared it to change scores and analysis of covariance (ANCOVA) in a two-group pretest-posttest design. Results indicated that the EIV method’s estimates were unbiased under many conditions, but the EIV method consistently demonstrated lower power than the change score method. An additional risk with using the EIV method is that one must enter the covariate reliability into the EIV model, and results highlighted that estimates are biased if a researcher chooses a value that differs from the true covariate reliability. Obtaining unbiased results also depended on sample size. Our conclusion is that there is no additional benefit to using the EIV method over change score or ANCOVA methods for comparing the amount of change in pretest-posttest designs.
References
(1990). Change scores as dependent variables in regression analysis. Sociological Methodology, 20, 93–114. doi: 10.2307/271083
(1978). Robustness? The British Journal of Mathematical and Statistical Psychology, 31, 144–152. doi: 10.1111/j.2044-8317.1978.tb00581.x
(2000). Structural equation models and the regression bias for measuring correlates of change. Educational and Psychological Measurement, 60, 893–907. doi: 10.1177/00131640021970970
(2004). Decreases in posttest variance and the measurement of change. Methods of Psychological Research, 9, 37–55.
(1970). How should we measure “change”: Or should we? Psychological Bulletin, 74, 68–80. doi: 10.1037/h0029382
(2011). Using analysis of covariance (ANCOVA) with fallible covariates. Psychological Methods, 16, 166–178. doi: 10.1037/a0023355
(2001). A conundrum in the analysis of change. Nutrition, 17, 360–361. doi: 10.1016/S0899-9007(00)00593-1
(1980). Properties of some estimators for the errors-in-variables model. Annals of Statistics, 8, 407–422. doi: 10.1214/aos/1176344961
(1987). Measurement error models. New York, NY: Wiley.
(1999). Dealing with baseline differences: Two principles and two dilemmas. International Journal of Psychophysiology, 31, 155–161. doi: 10.1016/S0167-8760(98)00048-8
(2004). Analysis of covariance (ANCOVA) with difference scores. International Journal of Psychophysiology, 52, 277–283. doi: 10.1016/j.ijpsycho.2003.12.009
(1977). The determination of the significance of change between pre- and posttesting periods. Review of Educational Research, 47, 121–150. doi: 10.1037/0022-006X.59.1.27
(1967). A paradox in the interpretation of group comparisons. Psychological Bulletin, 68, 304–305. doi: 10.1037/h0025105
(1998). Covariance adjustment versus gain scores-revisited. Psychological Methods, 3, 309–327. doi: 10.1037/1082-989X.3.3.309
(2001). Misunderstanding analysis of covariance. Journal of Abnormal Psychology, 110, 40–48. doi: 10.1037/0021-843X.110.1.40
(1987). Analysis of covariance: Its model and use in psychological research. Journal of Counseling Psychology, 34, 383–392. doi: 10.1037/0022-0167.34.4.383
. (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing [Computer software manual]. Vienna, Austria: Retrieved from http://www.R-project.org
(1988).
Myths about longitudinal research . In K. W. SchaieR. T. CampbellW. M. MeredithS. C. RawlingsEds., Methodological issues in aging research (pp. 171–209). New York, NY: Springer.(1995).
Myths and methods: “Myths about longitudinal research” plus supplemental questions . In J. M. GottmanEd., The analysis of change Mahwah, NJ: Erlbaum.(2006). Change from baseline and analysis of covariance revisited. Statistics in Medicine, 25, 4334–4344. doi: 10.1002/sim.2682
(1974). An errors-in-variables analysis of managerial role performance. Journal of the American Statistical Association, 69, 886–893. doi: 10.1080/01621459.1974.10480223
(2006). Comparing groups in a before-after design: When t test and ANCOVA produce different results. The British Journal of Educational Psychology, 76, 663–675. doi: 10.1348/000709905X52210
