Consistency and Efficiency of Ordinary Least Squares, Maximum Likelihood, and Three Type II Linear Regression Models
Abstract
Abstract. Type I linear regression models, which allow for measurement errors only in the criterion variable, are frequently used in modeling research in psychology and the social sciences. Although there are frequently measurement errors and large natural variation both in the criterion and predictor variables, type II regression methods that account for these errors are seldom used in these fields of study. The consistency and efficiency of three type II regression methods (reduced major axis, Kendall's robust line-fit and Bartlett's three-group) were evaluated in comparison to ordinary least squares (OLS) and the maximum likelihood with known variance ratio used frequently in biometrics and econometrics. When predictors are measured with error, OLS slope estimates are biased toward zero, and the same bias was observed with both Kendall's and Bartlett's methods. Reduced major axis produced consistent estimates even for small sample sizes, whenever the measurement errors in X are similar in magnitude to measurement errors in Y, but there was a consistent bias when the measurement error in X was smaller/greater than in Y. Maximum likelihood estimates behaved erroneously for small sample sizes, but for larger sample sizes they converged to the expected values.
References
Bollen, K.A. (1989). Structural equations with latent variables . New York: WileyCarrol, R.J. , Ruppert, D. (1996). The use and misuse of orthogonal regression in linear error-invariables models. American Statistician, 50, 1– 6Carrol, R.J. , Ruppert, D. , Stefanski, L.A. (1995). Measurement error models . New York: Wiley and SonsCheng, C.-L. , VanNess, J.W. (1999). Statistical regression with measurement error . London: ArnoldFreedman, L.S. , Fainberg, V. , Kipnis, V. , Midthune, D. , Carroll, R.J. (2004). A new method for dealing with measurement error in explanatory variables of regression models. Biometrics, 60(1), 172– 181Fuller, W.A. (1987). Measurement error models . New York: WileyIsaac, P.D. (1970). Linear regression, structural relations, and measurement error. Psychological Bulletin, 74, 213– 218Jaccard, J. , Wan, C.K. (1995). Measurement error in the analysis of interaction effects between continuous predictors using multiple regression: Multiple indicator and structural equation approaches. Psychological Bulletin, 117, 348– 357Jolicouer, P. (1975). Linear regressions in the fishery research: Some comments. Journal of the Canadian Fishery Research Board, 32, 1491– 1494Jöreskog, K.G. , Sörbom, D. (1982). Recent developments in structural equation modeling. Journal of Marketing Research, 19, 404– 416Kendall, M. , Stuart, A. (1961). The advanced theory of statistics . London: Charles Griffin & CompanyKendall, M.G. , Gibbons, J.D. (1990). Rank correlations methods (5th ed.). London: Edward ArnoldKlauer, K.C. , Draine, S.C. , Greenwald, A.G. (1998). An unbiased errors-in-variables approach to detecting unconscious cognition. British Journal of Mathematical and Statistical Psychology, 51, 253– 267Kuhry, B. , Marcus, L.F. (1977). Bivariate linear models in biometry. Systematic Zoology, 26, 201– 209Kulathinal, S.B. , Kuulasmaa, K. , Gasbara, D. (2002). Estimation of an errors-in-variables regression model when the variances of the measurement errors vary between the observations. Statistics in Medicine, 21, 1089– 1101Legendre, P. , Legendre, L. (1988). Numerical ecology (2nd English ed.). Amsterdam: Elsevier ScienceLi, T. (2002). Robust and consistent estimation of nonlinear errors-in-variables models. Journal of Econometrics, 10, 1– 26Marsh, H.W. , Hau, K.-T. , Balla, J.R. , Grayson, D. (1998). Is more ever too much: The number of indicators per factor in confirmatory factor analysis. Multivariate Behavioral Research, 33, 181– 220Miller, J. (2000). Measurement error in subliminal perception experiments: Simulation analyses of two regression methods. Journal of Experimental Psychology, 26, 1461– 1477Morton-Jones, T. , Hederson, R. (2000). Generalized least squares with ignored errors in variables. Technometrics, 42, 366– 375Pakes, A. (1982). On the asymptotic bias of the Wald-type estimators of a straight line when both variables are subject to error. International Economics Review, 23, 491– 497Quinn, G.P. , Keough, M.J. (2002). Experimental data analysis for biologists . Cambridge: Cambridge University PressRayner, J.M.V. (1985). Linear relationships in biomechanics: The statistics of scaling functions. Journal of Zoology (London), Series A, 206, 415– 439Ricker, W.E. (1973). Linear regression in the fishery research. Journal of Fisheries Research Board of Canada, 30, 409– 434Ricker, W.E. (1984). Computation and uses of central trend lines. Canadian Journal of Zoology, 62, 1897– 1905Riggs, D.S. , Guarnieri, J.A. , Addelman, S. (1978). Fitting straight lines when both variables are subject to error. Life Sciences, 22, 1305– 1360Rock, D.A. , Werts, C.E. , Linn, R.L. , Jöreskog, K.G. (1977). A maximum likelihood solution to the errors in variables and errors in equations models. Multivariate Behavioral Research, 12, 187– 197Ryu, H.K. (2004). Rectangular regression for an errors-in-variables model. Economics Letters, 83, 129– 135Schuster, C. (2004). How measurement error in dichotomous predictors affects the analysis of continuous criteria. Psychology Science, 46(1), 128– 136Sokal, R.R. , Rohlf, F.J. (1995). Biometry. The principles and practice of statistics in biological research (3rd ed.). New York: FreemanTheil, H. (1950). A rank-invariant method for linear and polynomial regression analysis. Indagationes Mathematicae, 12, 85– 91Weston, R. , Gore, P.A.J. (2006). A brief guide to structural equation modeling. Counseling Psychologist, 34, 719– 751
