On Correlated Errors

Whether errors can be correlated is an old matter that is as yet unsettled. The literature reveals that the discussion concerning correlated errors began some 80 years ago (e.g., Aitken, 1934). In the framework of confirmatory factor analysis this question has even become ubiquitous since correlated errors developed into the preferred means of overcoming problems concerning model fit. Virtually every ill-fitting model can be transformed into a well-fitting one by allowing pairs of error components to correlate with each other. The larger the number of pairs of correlated error components, the better the model fit usually is.

Although I am not encouraging the use of correlated errors (Schweizer, 2010), a number of manuscripts reporting correlated errors were submitted to the European Journal of Psychological Assessment in the past – and some of them were in fact published. The issues of the last 2 years include eight papers (De Carvalho, Leite, Seminotti, Freitas, & de Lourdes Drachler, 2011; Derkman, Scholte, Van der Veld, & Markland, 2010; Di Giunta, Eisenberg, Kupfer, Steca, Tramontano, & Caprara, 2010; Höfling, Moosbrugger, Schermelleh-Engel, & Heidenreich, 2011; Mumma, 2011; Teubert & Pinquart, 2011; Veirman, Brouwers, & Fontaine, 2011; Zohar, Denollet, Ari, & Cloninger, 2011) that report models including correlated error components. It should be added that the numbers of correlated error components reported in most articles are small. The publication of these papers gratifies scientific honesty in a world that suffers from publication pressure. Furthermore, it should be added that my investigation of the EJPA issues even revealed one article including the statement that “correlated errors ... were not permitted” (Houghton, Durkin, Ang, Taylor, & Brandtman, 2011).

However, the use of correlated errors is risky with respect to validity because of the damage that is done to the model. The likelihood is high that such a model no longer sufficiently reflects the concept it is assumed to represent. If a model with correlated errors is applied for test construction, the structural validity of the measure is likely to be impaired. Reasons excluding such damage are seldom given. There may be cases where a characteristic of a specific sample or simply chance justifies correlated errors; however, such cases are really rare (Zimmerman & Williams, 1977).

From the perspective of philosophy of science correlated errors are a disaster since they in no way contribute to our knowledge. Including correlated errors into a model does not create a revised model that may be considered as an alternative to the original model. Although the revised model would be a posthoc model, it would be a valuable model since it would be a specific model, i.e., a model that contributes to knowledge. Instead, correlated errors simply weaken the testability of the model. One can no longer increase our knowledge by disproving the hypothesis, i.e., the major hypothesis underlying the model (Popper, 1934), respectively by contributing to a fair balance of favorable and unfavorable findings concerning the hypothesis that can provide a reasonable basis for progress in science (Lakatos & Musgrave, 1970).

In many cases the necessity of correlated errors simply signifies that the model does not consider all the important characteristics of the data. There are established approaches to dealing with this problem: First, one can consider a possible substructure. For example, in the case of GENOS as measures of emotional intelligence (recently published in EJPA, see Gignac, 2010), consideration of a substructure led to an acceptable degree of model fit. Second, measurement impurity can cause model misfit; consideration of the source of impurity as part of the model can lead to a good degree of model fit otherwise not possible (Schweizer, 2007). Third, a method effect may have to be taken into consideration, for example, by considering method factors (Schroeders & Wilhelm, 2010). In each of these cases exemplifying the approaches there is a modification of the original model, and this modification increases our knowledge about human nature.

Finally, one can avoid correlated errors by computing parcels. Including the crucial items in a parcel may solve the problem of model misfit, but it may also hide something we need to know when evaluating a measure. So computing parcels may not truly contribute to establishing structural validity. However, the computation of parcels can be reasonable if a very large number of items has to be considered in an investigation of structural validity (e.g., Gorostiaga, Balluerka, Alonso-Arbiol, & Haranburu, 2011; Lehmann-Willenbrock, Grohmann, & Kauffeld, 2011).