Causal Effects Based on Latent Variable Models
Abstract
Abstract. Building on the stochastic theory of causal effects and latent state-trait theory, this article shows how a comprehensive analysis of the effects of interventions can be conducted based on latent variable models. The proposed approach offers new ways to evaluate the differential effects of interventions for substantive researchers in experimental and observational studies while allowing for complex measurement models. The key definitions and assumptions of the stochastic theory of causal effects are first introduced and then four statistical models that can be used to estimate various types of causal effects with latent state-trait models are developed and illustrated: The multistate effect model with and without method factors, the true-change effect model, and the multitrait effect model. All effect models with latent variables are implemented based on multigroup structural equation modeling with the EffectLiteR approach. Particular emphasis is placed on the development of models with interactions that allow for interindividual differences in treatment effects based on latent variables. Open source software code is provided for all models.
In the last decades, theories of causal effects (e.g., Pearl, 2009; Rubin, 1974; Steyer, Mayer, & Fiege, 2014) and theories of latent variables (see, e.g., Bollen, 2002; Hambleton & Swaminathan, 1985; Lord & Novick, 1968; Steyer, Mayer, Geiser, & Cole, 2015) have been developed and have made important contributions to science in general. Such theories are very important (not only) for the social and behavioral sciences, because they provide us with clear non-model based definitions of the quantities of interest such as average causal effects or values of latent variables and clarify the assumptions that are necessary to estimate these quantities using statistical models.
The introduction of these theories has subsequently led to a wide variety of statistical approaches that can be used to make inferences about the core theoretical concepts from empirical studies. In the causal inference literature there are regression adjustment techniques (Shadish, Cook, & Campbell, 2002), methods based on the propensity score (PS; Rosenbaum & Rubin, 1983) such as PS weighting, PS matching, PS adjustment, or PS stratification (Imbens & Rubin, 2015; Stuart, 2004), methods based on instrumental variables (Angrist, Imbens, & Rubin, 1996; Heckman, 1997), and doubly robust approaches combining multiple methods (Robins, Rotnitzky, & Zhao, 1994; Kang & Schafer, 2006). For an overview, see Schafer and Kang (2008).
In the latent variable literature and especially in psychometrics, there are quite a lot of models that can be used to measure latent variables, ranging from classic confirmatory factor analysis to multitrait-multimethod models (Eid, 2000; Marsh & Grayson, 1995), to models with method factors (see Geiser & Lockhart, 2012), to longitudinal state-trait models (Steyer, Ferring, & Schmitt, 1992; Steyer et al., 2015), to bifactor models (Eid, Geiser, Koch, & Heene, 2017; Holzinger & Swineford, 1937). Most of these models have been presented for continuous and for categorical indicators of latent variables and have also been extended to longitudinal designs.
While there is some work on the causal relationship between the indicators and the latent constructs (Bollen, 2002), surprisingly little has been written about how to estimate causal effects using latent variable models. Some exceptions based on relatively simple latent variable models are given by Mayer, Dietzfelbinger, Rosseel, and Steyer (2016) and Sengewald, Steiner, and Pohl (2018). This article aims at closing this gap and shows how to estimate causal effects based on more complex latent variable models with customized measurement models using the EffectLiteR approach – a multigroup structural equation modeling approach for the analysis of average and conditional effects presented by Mayer et al. (2016). Particular emphasis is placed on the development of models with interactions that allow for interindividual differences in treatment effects based on latent variables.
Incorporating measurement models for latent variables when examining treatment effects can be crucial. A fallible covariate can cause bias in estimation of regression coefficients (Carroll, Ruppert, Stefanski, & Crainiceanu, 2010) and thereby also in average and conditional effects. The regression coefficient of the fallible covariate is biased toward zero. This bias may not only affect the coefficient for the covariate itself, but also the coefficients for other manifest or latent covariates and the treatment (e.g., Wooldridge, 2010). Since the regression coefficient of the fallible covariate is biased toward zero, it is also more difficult to identify covariates which significantly affect the treatment or the outcome when measurement error is not appropriately taken into account. Sengewald et al. (2018) identified several conditions under which controlling for fallible covariates leads to substantial bias in average effects. Even in a randomized experiment, where omitting the latent covariate does not introduce bias, it may still be beneficial to consider the latent covariate, because it increases power. Measurement error in the outcome variable does not lead to biased estimates of causal effects, unless measurement error variables are correlated with both, the treatment variable and covariates. However, incorporating measurement models for the outcome variable can also be beneficial, because it may lead to more adequate estimates of effect sizes.
This article can be viewed as a follow-up of Steyer’s (2005) article in which he described some latent variable models for analyzing individual causal effects, but goes beyond in several respects: First, the theory of causal effects is updated and time is introduced explicitly following Steyer et al. (2014). Second, the no treatment – no change assumption is relaxed and causal effects are defined as differences between the treatment group and the control group. Third, it is shown how to include covariates in the analysis including categorical covariates and propensity scores. Fourth, open source software and code for computing average and conditional causal effects based on latent variable models is provided in Appendices A to D. The paper is structured as follows: The stochastic theory of causal effects and the EffectLiteR approach are introduced. Then various examples are shown, how this approach can be used to estimate causal effects based on multistate models with and without method factors, true-change models, and multitrait models.
Stochastic Theory of Causal Effects
Definitions of Causal Effects
Most theories of causal effects start with defining (individual) causal effects (e.g., Imbens & Rubin, 2015; Rubin, 1974; Pearl, 2009; Steyer et al., 2014). The key idea of defining an individual treatment effect of a treatment X = 1 versus another treatment X = 0 is to compare the expected outcome under both treatment conditions for a single person. Considering time is crucial when defining a causal effect. Obviously, the treatment variable X has to be prior to the outcome variable η. Steyer et al. (2014) view causality as a stochastic process and explicitly refer to the time point when random variables occur for the first time using filtrations. In probability theory, such filtrations play an important role in the formalization of stochastic processes and are used to model the information that is available at a given time point. Formally, filtrations are ordered families of σ-algebras that contain the history of the stochastic process up to a given time point, that is, that are generated by the random variables up to a given time point (Klenke, 2008). In this article, a simplified version of this theory is presented. To illustrate the idea of a stochastic process consider Figure 1. Every person at a given time point t has a history that contains all the previous experiences up to this time point, as in latent state-trait theory (Steyer et al., 2015). The random experiment starts at a time point t0. Let the person variable at this time point be denoted by Ut0. As time goes on, persons evolve and make new experiences that become part of them. The crucial time point for defining a (total) causal effect is the time point tX when X occurs. The person variable just before this time point is denoted by Utx and all variables that are measurable at this point in time (except for X itself) are termed covariates.1 Covariates can be categorized along two dimensions – they can either be categorical or continuous or manifest or latent. Categorical covariates are denoted by K and continuous covariates by the vector ξ. When considering multiple categorical covariates K1, K2, …, the values of the single unfolded categorical covariate K represent all possible combinations of values of multiple categorical covariates. Figure 1 shows an exemplary latent covariate ξ measured by three indicators and some happy and sad experiences that may affect the person over time. At time point tX the person is either assigned to one of two treatment conditions with a certain probability (randomized experiment) or chooses a treatment condition (observational study). Later on at the time point tη a value of the outcome variable η will be observed. This value is not fixed a priori at the beginning of the random experiment, that is, an intra-individual distribution is assumed under both treatment conditions as illustrated by the bell-shaped curves in Figure 1 (This presentation follows Steyer’s theory and uses stochastic true outcomes instead of Rubin’s fixed potential outcomes. See Steyer, 2005, for a discussion). Let τx = E(η|X = x, Utx) denote the true outcome variable for persons just before the treatment Utx under treatment X = x. The individual treatment effect variable can then be defined as the difference between two true outcome variables:
Causality Conditions and Identification
Ideally it would be possible to directly estimate the individual treatment effects in an empirical study. However, this is usually not possible because a person can only be observed once under one of the treatment condition. In theory, if it would be possible to observe every person multiple times under each of the treatment conditions, then the intra-individual distribution under each treatment could directly be estimated, and unbiased estimates for the individual treatment effects as defined above could be obtained. The only situations when it is actually possible to estimate individual treatment effects from randomized controlled trials or observational studies are when there is no pre-post change at all in the control group (Steyer, 2005) or when it can be assumed that all covariates explaining unique variance in the outcome variable have been included in the analysis. Steyer, Gabler, von Davier, and Nachtigall (2000) term the latter causality condition conditional unit-treatment homogeneity. Both of these causality conditions are pretty strong conditions. The discussion section elaborates more on these assumptions and on their plausibility in empirical studies.
Fortunately, even when the individual effects cannot directly be estimated, aggregates of individual effects defined in Table 1 can still be estimated under weaker causality conditions. The starting point is the regression E(η|X, K, ξ) of the dependent variable on the treatment X, categorical covariates K, and continuous covariates ξ. In the statistical models for analyzing causal effects, the conditional effect function g1(K, ξ) always plays a central role. It is the function whose values are the conditional effects of treatment X = 1 compared to the control group X = 0 defined as:
The conditional effect function, as opposed to the individual causal effect variable, does not contain the person variable Utx but only covariates K and ξ that are functions of the person variable, that is, the individual effects are aggregated at the level of the covariates. When the conditional effects are unbiased, this expression can then be used to identify average and conditional causal effects as shown in Table 1. Unbiasedness of g1(K, ξ) means that its values are equal to the corresponding conditional causal effects. More formally, unbiasedness of the effect function is given when the following causality condition holds:
This condition is usually not testable, because δ10 can not be estimated directly. However, unbiasedness of the effect function is implied by other, more well-known causality conditions such as randomization, conditional randomization, conditional unit-treatment homogeneity, ignorability, and others (Steyer et al., 2000; Rosenbaum & Rubin, 1983). Table 2 shows some causality conditions that imply unbiasedness of the effect function, of which some are testable in the sense that they can be falsified based on empirical evidence (see, e.g., Mayer, Thoemmes, Rose, Steyer, & West, 2014). The causality conditions either refer to (conditional) stochastic independence of treatment and persons or to conditional mean independence of the dependent variable and persons. In particular, the stochastic independence causality conditions mean that treatment assignment is independent of persons (randomization), that treatment assignment is independent of persons given the measured categorical and continuous covariates (conditional randomization), and that treatment assignment is independent of the true outcome variables given the measured categorical and continuous covariates (ignorability). The mean independence causality condition means that the dependent variable is regressively independent of the person, given the treatment condition and the measured categorical and continuous covariates (conditional unit-treatment homogeneity). Unbiasedness can also be deduced based on directed acyclic graphs using the graphical back-door criterion (Pearl, 2009) that can inform us, if it is sufficient to condition on K and ξ given the graph is correctly specified. Once having an unbiased conditional effects function, it is possible to again aggregate and consider aggregated effects at a higher level, such as average effects, effects for subgroups or effects given a subset of all relevant covariates. Table 1 shows definitions for various causal effects and the identification based on an unbiased conditional effects function.
Models for Causal Effects
Statistical Models
In order to estimate average and conditional causal effects based on data from an empirical study, a statistical model is needed to estimate the conditional effects function g1(K, ξ). Researchers can use a wide range of statistical methods for this purpose: Ordinary least squares regression with or without interactions (Aiken & West, 1991), generalized linear models with different link functions (McCullagh & Nelder, 1989), multilevel models (Raudenbush & Bryk, 2002), semi-parametric models (Hastie & Tibshirani, 1990) or models from the machine learning literature such as recursive partitioning or random forests (Hastie, Tibshirani, & Friedman, 2001) to name just a few. In principle, every suitable statistical model can be used to analyze causal effects. Of course these statistical models do not per se yield causal effects, only if unbiasedness of the conditional effects function is fulfilled can the effects be interpreted in a causal manner. None of the aforementioned approaches can easily incorporate latent variables, which are very common in the social and behavioral sciences. Usually, structural equation models (Bollen, 1989) are used as statistical models with latent variable but they are mostly applied in the linear setting. In this paper, the EffectLiteR approach will be used to estimate causal effects in complex latent variable models with higher-order interactions. The EffectLiteR approach is a multigroup structural equation modeling approach with stochastic group sizes and has been developed by Mayer et al. (2016) and Mayer and Thoemmes (2019) building on previous work by Kröhne (2009) and Steyer and Partchev (2008).
Causal Inference Techniques
Including covariates in the conditional effects function can serve two purposes: First, covariates may be included to learn more about the differential effects of the treatment or intervention, that is, to learn how the treatment effects depend on the values of the included covariates. Second, in observational studies it is necessary to include potential confounding variables such that unbiasedness of the conditional effect function holds. This is sometimes also called regression adjustment (Shadish et al., 2002). In addition to regression adjustment, the causal inference literature has brought forward the development of causal techniques that can aid in the adjustment. These techniques are mostly based either on the propensity score (PS; Rosenbaum & Rubin, 1983) or on instrumental variables (Angrist et al., 1996). The propensity score represents the individual treatment probability defined as P(X = 1|Utx) and is usually estimated using logistic regression of the treatment variable on a vector of covariates. In particular the propensity score has been helpful in adjusting for a broad set of potential confounding variables that can be summarized in the propensity score. In the causal analysis, the propensity score can then be used in weighting, matching, and stratification approaches. The EffectLiteR approach can also be combined with these causal techniques as will be discussed later in this article. Within this context of latent variable models, one possible option is to include important latent covariates in the statistical model and additionally include the logit-transformed propensity score based on manifest variables as additional covariate. Such approaches that combine two adjustment techniques have been termed doubly robust methods (for an overview, see Schafer & Kang, 2008).
The EffectLiteR Approach
The EffectLiteR approach is a comprehensive framework for evaluating the differential effects of a treatment based on latent variable models. Its core component is a multigroup structural equation model with stochastic group sizes. The parameters of the conditional effect function g1(K, ξ) as well as all average and conditional causal effects are then computed based on the parameters of the multigroup structural equation model. Different parameterizations for the effect function can be considered as well as different estimators. By default full information maximum likelihood is used and parameters and effects are computed via the multivariate delta method (for details, see Mayer et al., 2016).
Applications of EffectLiteR
The EffectLiteR approach has been successfully applied to analyze average and conditional effects in various substantive examples especially from clinical psychology and educational science. For example, it has been used to estimate the effects of an autonomy-supportive intervention for teaching physics lessons in high school (Flunger, Mayer, & Umbach, 2018; Mayer, Umbach, Flunger, & Kelava, 2017), for analyzing the effects of inpatient psychotherapy on attachment characteristics (Kirchmann et al., 2011), for examining early transition to secondary school compared to late transition (Mayer, Nagengast, Fletcher, & Steyer, 2014), for evaluating a stress prevention program with teacher students (Karing & Beelmann, 2016), and for looking at differential effects of reading trainings on reading processes (Müller et al., 2015). Most of these examples include either latent covariates or latent outcome variables in the statistical model. Next, the general underlying statistical models will be briefly described and later some examples are provided in more detail.
Multigroup SEM With Stochastic Group Sizes
As mentioned above, the underlying statistical model for the EffectLiteR approach is a multigroup structural equation model (multigroup SEM) with stochastic group sizes. Building upon multigroup SEM has several advantages such as the possibility to consider latent dependent variables and covariates, the modeling of stochastic regressors, the option to include higher-order interactions, and the incorporation of many recent developments from the SEM literature. The multigroup SEM consists of a group-invariant measurement model relating manifest variables in the vector y to latent variables in the vector η, a group-specific structural model specifying the regressions among latent variables, and a model for the group sizes:
Effect Function
The basis for estimating all average and conditional causal effects is the conditional effect function. So far, the particular functional form for the effect function has not yet been specified. In the EffectLiteR approach, the default parameterization of the effect function comparing treatment from X = 1 to X = 0 includes higher-order interactions and takes the following functional form:
Effect Models With Latent Variables
Now some examples will be described with increasing complexity for specific submodels of the EffectLiteR approach that illustrate the procedure. For didactic reasons, the first model is a repetition of the multistate effect model that has been presented by Mayer et al. (2016), but does not include categorical covariates. Then three effect models with latent variables are introduced that have not yet been described in detail in the literature: (1) The true-change effect model and it’s relation to the multistate effect model; (2) the multistate effect model with method factors, logit-transformed propensity score and categorical covariate; and (3) the multitrait effect model with method factors, logit-transformed propensity score and categorical covariate. The latter model is an extension of the simpler version presented by Steyer et al. (2015). All models are presented with model equations, path models, and computations. The accompanying software code for the corresponding model is given in Appendices A to D.
Multistate Effect Model
The simplest presented model is a multistate effect model. A similar example has also been described in part by Steyer and Partchev (2008) in the user’s manual of the predecessor of the EffectLiteR program and by Mayer et al. (2016) who extended the approach to include categorical covariates. The basis for this model is a two-group multistate model for two occasions of measurement (Steyer et al., 2015. This corresponds to a classic pretest-posttest design with a treatment group X = 1 and a control group X = 0. The latent pretest variable and the latent outcome variable are both measured by two indicators Yit, where i denotes the item and t the time point. An essentially τ-equivalent measurement model is assumed, that is, group-invariant and the scale of the latent variables are fixed by setting the first intercept to zero. The treatment assignment is randomized, consequently, it is not required to control for the latent pretest or other covariates from an adjustment perspective, but it is still informative in order to investigate the dependence of the effect function from the latent pretest and to increase power. The group-specific model equations for the SEM are given by:
Following latent state-trait theory, group-invariance of the measurement model is required in the EffectLiteR approach but not necessarily longitudinal-invariance. The latter assumption could be relaxed if needed. A path diagram for the group-specific SEM is shown in Figure 2 and the EffectLiteR code for specifying the model is given in Appendix A. The EffectLiteR code allows the specification of the measurement model in lavaan code (Rosseel, 2012) taking into account the number of groups and then the main effectLite() function is called with the desired arguments. The mean structure for the latent variables will be specified automatically by EffectLiteR. There is also a function generateMeasurementModel() that aids in the specification of the measurement model, but since longitudinal invariance is assumed in this example, a customized measurement model is needed. The next step is to compute the conditional effects function g1(ξ) from the parameters of the multigroup SEM:
The effect function is the difference between the two group-specific regressions: γ10 is the difference of the group-specific intercepts and γ11 is the difference of the group-specific slopes, that is, the interaction term. Based on the effect function and μx, the conditional means of ξ, the average and conditional effects are identified (cf. Table 1):
True-Change Effect Model
Some researchers intuitively think that it makes more sense to consider the difference between the posttest and the pretest as dependent variable instead of the posttest itself. In this article, it is argued that this can be done but yields identical results as with the multistate effect model, a result that is well-known in the methodological literature on manifest change score variables (e.g., Kessler & Greenberg, 1981). The reason can easily be seen by considering the following two regression equations:
Subtracting ξ on both sides yields the equation for the change version:
So, when the pretest ξ is included, both equations yield the identical coefficients β2 for X and β3 for the interaction term Xξ. Only the coefficient for ξ is changing and becomes (β1 − 1) in the change model. Note that including ξ as regressor in the change score equation is necessary from the standpoint of the stochastic theory of causal effects, since all variables that are prior to the treatment are controlled for in defining causal effects. In randomized experiments, the regressions E(η|X) and E(η – ξ|X) will also yield unbiased effects.
The EffectLiteR approach nevertheless can be used to examine effects on the latent change variable. This model with the change variable as dependent variable is called a true-change effect model, because its basis is a multigroup true-change model (Steyer, Eid, & Schwenkmezger, 1997). The model equations are similar to the multistate effect model, except that the loadings of the latent intercept factor now refer to all manifest variables, which changes the meaning of the latent outcome variable to represent a latent change score π = η − ξ:
A path diagram for the group-specific true-change effect model is shown in Figure 3 and the EffectLiteR code for specifying the model is given in Appendix B. As mentioned above, the true-change effect model will give identical point estimates and standard errors for all average and conditional effects of X as the multistate effect model. Also the residual standard deviations will be identical. However, there are two noteworthy differences between the two formulations: First, the effect size will be different, since for obtaining the effect size, the point estimates are divided by the standard deviation of the dependent variable, that is, η in the multistate effect model and π in the true-change effect model. The default setting in EffectLiteR is to divide by the standard deviation of the dependent variable in the control group. In many cases, the variance of the change variable is lower than the variance of the pretest, leading to seemingly higher effect sizes in the true-change effect model. In general, the following well-known formula describes the relationship between the two variances:
Steyer (2005) has also suggested to use a true-change model to analyze individual causal effects, but his model is different from the one presented in this article. Steyer’s approach requires the additional assumption that there is no change at all in the control group except for measurement error. So he uses a true-change model in the treatment group and a stable model in the control group, whereas in this article a true-change model is assumed in both groups and differences in change between the two groups are considered.
Multistate Effect Model With Method Factors
Returning to the simple multistate model introduced earlier for didactic reasons, such a model is oftentimes too simple and practical situations require extensions and more advanced models. For example, in observational studies it is necessary to include additional categorical and continuous covariates to obtain unbiased effects. Other causal effects such as effects on the treated may be more interesting in such situations. And the measurement model may require method factors or correlated residuals since the same items are used multiple times. Such a more complex model, with additional continuous covariates, propensity scores, method factors, and effects on the treated, has been used in the study by Kirchmann et al. (2011), but has not been described in detail due to space limitations and the substantive focus of the article. Here the details of the model are described along with the model equations, the path model, the specification in EffectLiteR and also extend it to categorical covariates so that it can be more easily adapted by researchers.
In longitudinal studies, the same indicators are presented repeatedly over time. In practice, the assumption of strict parallelism of the indicators within a time point rarely holds and the indicators oftentimes have shared indicator-specific variance over time (method variance). Multiple approaches have been suggested to deal with this problem ranging from correlated measurement error variables to various method factor models (Geiser & Lockhart, 2012). Due to the flexibility in specifying the measurement model, the EffectLiteR approach can in principle be used in combination with all these approaches. In this article, it has been shown how method factors defined as differences between true score variables (Pohl, Steyer, & Kraus, 2008) can be used in an effect analysis model. The corresponding measurement model then is:
The first indicator serves as reference indicator and the method factor contains the differences between the true scores of the second and the first indicator. In contrast to the multistate effect model with essentially τ-equivalent measures, the intercepts in the measurement model are now all fixed to zero, since systematic differences between the indicators are captured by the method factor whose mean and variance are freely estimated in all groups. A key question is whether the method factor should be added as a covariate in the effect model or not. From the perspective of the stochastic theory of causal effects, all confounding variables that are prior to the treatment variable need to be added. The method factor is defined as difference between the two true score variables within each time point and it is assumed that the method effects are constant over time, that is, 
. Because the method effects are invariant over time, the method factor is already measurable at the time point of the pretest and should therefore be included as a covariate in the model in this approach.
In an observational study, controlling for the pretest and method effects is usually not enough and other continuous and categorical covariates need also be included such that unbiasedness of the effect function holds. One way to summarize a list of covariates Z1, Z2, …, Zq is by using a logit-transformed propensity score instead of the single covariates. The propensity score represents the treatment probabilities and can for example be estimated using a logistic regression:
The logit-transformed propensity score Zprop can then be included as covariate in the EffectLiteR model. Adding latent covariates to the propensity score is less straightforward and can, for example, be achieved with factor scores (Raykov, 2012; Sengewald et al., 2018). In the equation above, a simple additive model is assumed for the propensity score. The selection of covariates and interactions, the specification of the functional form for the logistic regression, or balancing checks for the propensity are not discussed. When including propensity scores in practice, these issues need more attention, but a detailed discussion is beyond the scope of this article (see Guo & Fraser, 2010, for details).
In order to additionally control for important categorical covariates, the multigroup SEM approach in EffectLiteR can be extended and categorical covariates can directly be incorporated into the statistical model. In this case, the groups are formed by the treatment groups X and values of categorical covariates K, so there are as many groups as there are unique combinations of values of X and K. In an example with two treatment groups and a binary categorical variable K, the underlying statistical model turns into a four group SEM, where the regression of the dependent variable on the latent pretest, the method factor, and the propensity score is group-specific, resulting in the following equation for the structural model in group (X = x, K = k):
The model for the group sizes remains the same and is not repeated. Notice that the group sizes are required to compute expectations and conditional expectations for categorical and continuous covariates and must therefore be included. A path diagram for the complete group-specific multistate effect model with method factors is shown in Figure 4 and the EffectLiteR code for specifying the model is given in Appendix C. The default conditional effects function g1(K, ξ) in the EffectLiteR approach includes two- and three-way interactions and takes the following functional form:
The γ1kq parameters of the effect function are computed based on the αxkq parameters of the multigroup SEM. Some γ1kq coefficients are differences between αxkq parameters and other γ1kq coefficients are differences of differences of αxkq parameters:
These expressions can be computed automatically with the EffectLiteR software package (see Appendix C) and the general formulas are derived in Mayer et al. (2016). The package also computes all average and conditional effects defined in Table 1 based on the model. For example, the conditional effect for K = 0 is computed as:
Multitrait Effect Model
The last model discussed in this article is the so-called multitrait effect model. Steyer et al. (2015) presented a similar multigroup-multistate-multitrait model for analyzing long-term effects of a treatment or an intervention that are not due to situational fluctuations and also do not make the restrictive assumption of no naturally occurring trait change in the control group as in Steyer (2005). Technically, these models build on a latent state-trait model and use a trait as the dependent variable in the effect analysis. As all the other models, they can be used in randomized controlled trials and in observational studies provided that unbiasedness of the effect function given the selected covariates holds. The model presented here goes beyond the previously described models in various aspects: The multigroup approach is extended to categorical covariates, propensity scores and method factors are included, and EffectLiteR Code is provided that is much more convenient and extensible than the manual specification of the multigroup-multistate-multitrait model in Steyer et al. (2015). For the multitrait effect model, a design with at least four occasions of measurement is needed, two before the treatment takes place and two after the treatment. Otherwise it is not possible to disentangle treatment effects and situational effects.
The measurement model is a second-order factor model. For this model, it is convenient to use separate equations for the first-order and the second-order parts of the measurement model:
The structural model and the model for the group sizes are identical as in the multistate effect model with method factors and are not repeated here. A path diagram for the group-specific multitrait effect model is shown in Figure 5 and the EffectLiteR code for specifying the model is given in Appendix D. The effect function and the computation of average and conditional effects is also identical as in the multistate effect model, only that the η and ξ variables now represent latent traits that are purged from situational effects as opposed to latent states that are a combination of trait and state residual components. This last example nicely shows the possibilities that the EffectLiteR approach offers for analyzing causal effects in experimental and quasi-experimental designs taking into account fairly complex measurement models for the covariates and the outcome of primary interest.
Discussion
Building on the theory of causal effects and latent state-trait theory, this article shows how a comprehensive analysis of the effects of interventions can be conducted based on latent variable models. The proposed approach offers new ways to evaluate the differential effects of interventions for substantive researchers in experimental and observational studies while allowing for complex first- and second-order measurement models with and without method factors. It brings together the statistical techniques developed in the causal inference literature, such as regression adjustment and propensity score modeling, and measurement models developed in psychometrics for measuring traits and states. If unbiasedness of the conditional effect functions holds, the approach yields average causal effects and various conditional causal effects. Four models have been presented in this article that can be considered special cases of the general EffectLiteR approach and can all be specified in the corresponding open source R package: The multistate effect model with and without method factors and additional covariates, the true-change effect model, and the multitrait effect model. In addition to the incorporation of latent variables, the EffectLiteR approach allows for higher-order interactions of treatments and covariates, which are useful for obtaining a complete picture of the effects of interventions. Limitations of the EffectLiteR approach are discussed in the practical recommendations section in Mayer et al. (2016). In particular, topics such as model complexity and higher-order interactions, sample size, missing data and sparse cells, and the use of large sample statistics are covered in this section and are not repeated here.
Measurement Invariance
The latent variable effect models presented in this article and the EffectLiteR approach in general require measurement invariance across groups. It is assumed that the intercept and the loadings of the measurement models are invariant. Widaman and Reise (1997) call this strong invariance. At first, this may seem to be a strong assumption and would limit the applicability of the EffectLiteR approach. However, the measurement invariance assumption is not only necessary in latent variable effect models, it is also inherent in models with manifest variables that are used for analyzing causal effects. In fact also manifest variable effect models make this assumption, but in these models, it is usually not tested and taken for granted that measurement invariance holds. With latent variable models, we can test the assumption and question our results if it does not hold. If measurement invariance is not plausible we may need to change our test, either by removing items or subscales that show differential item functioning, or by using another reliable test that is constructed in such a way that it can be used for fair comparisons between groups. This consideration is important irrespective of whether we use manifest or latent variable models.
While violations of measurement invariance across groups could lead to biased estimates of average and conditional effects, longitudinal invariance is not necessarily required. As shown in this article for the multistate effect model, longitudinal invariance can be imposed by using a customized measurement model. However, longitudinal invariance is not necessary for the EffectLiteR approach, because no mean comparisons between the pre- and posttreatment occasions are made. Instead the approach controls for the pretest just like it controls for the other covariates and therefore it is not necessary that the pretest and the posttest are on the same scale. If longitudinal invariance holds, it can be useful to implement the model with longitudinal invariance, since the estimates of the loadings then are more stable.
Individual Causal Effects
Recently, more and more researchers across different disciplines started focusing on how the effects of the treatment varies across persons in addition to the average effect. Especially in medicine and psychotherapy research, there is a lot of attention for such interindividual differences in responses to treatments. Under the heading of personalized medicine or precision medicine, researchers aim at identifying individual causal effects. Of course, estimating individual effects is the silver bullet of causal inference, but it requires strong designs and assumptions. In principle, the EffectLiteR approach can be used for estimating individual treatment effects when unit-treatment homogeneity (η ├ Utx|X, K, ξ) holds. In informal terms, unit-treatment homogeneity means that all covariates explaining unique variance in the outcome variable have been included in the analysis. This is a strong claim and can never be verified, because an important source of variation may have been missed in the data collection or the study design. But with careful designs, covariate selection procedures and techniques for identifying the functional form of the conditional effects function, it is possible to get close to the ideal of estimating individual effects in EffectLiteR.
The second strategy that allows for estimating individual causal effects is in within-subject designs where all alternative explanations for change can be ruled out. This approach has been described by Steyer (2005). These models require that there is no pre-post change at all in the control group except for measurement error in multistate models and situational effects in trait-change models. This assumption can actually be tested by using a stable model in the control group that needs to fit well. If the no change assumption holds in the control group, the true-change variable in the treatment group reflects the individual causal effects. However, the no change assumption is quite strong in many empirical examples. Even if the control group is a true control group where nothing happens, symptoms may get worse or improve by itself without any intervention over the course of the study. When the “control group” represents another treatment or a placebo group, the no change assumption usually is less plausible and the approach presented in this article of using a multistate effect model or a true-change effect model may be better suited.
Other Latent Variable Effect Models
In the social and behavioral sciences, measurement models and psychological tests play an important role. Human beings are notoriously difficult to measure – they are aware of test situations, they remember items and questions, they may be in social situations while taking the test, they respond differently to the same items, they change over time and so forth. Therefore measurement models and carefully constructed and validated tests are so important to measure latent characteristics of persons such as traits, states, emotions, and cognitions. So it is not surprising that a wide range of measurement models have been developed and in this article it is argued that these measurement models should also be included in the analysis of the effects of an intervention. The four models presented are only a small selection of possible measurement models in the EffectLiteR approach. In the future, this approach could also be combined with other latent variable models. For example, it would be interesting to consider tests with a bifactor structure to examine intervention effects on general and specific factors while controlling for covariates and specific factors at previous occasions. Or it could be used in combination with growth curve models with multiple posttreatment measurement occasions. Then the intervention may affect the posttreatment intercept and also the posttreatment slope and these effects may depend on multiple pretreatment covariates. To sum up, this article provides the methods for combining causal effects and latent variables and there is hope that more and more researchers make use of this idea and provide further methodological and substantive examples.
Since 2016, Axel Mayer is Assistant Professor for Psychological Methods at the Institute of Psychology at RWTH Aachen University. He received his PhD from Friedrich Schiller University Jena and has worked as a postdoctoral-doctoral researcher at the Department of Data Analysis at Ghent University.
1Utx is the person variable just before the treatment takes place. So it contains all person characteristics and experiences up to the treatment, but not the treatment itself. Steyer, Mayer, and Fiege (2014) present a slightly more general version that requires the introduction of a global covariate CX and additionally allows for covariates that are simultaneous to X and covariates that are not functions of Utx.
References
(1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.
(1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91, 444–455. https://doi.org/10.1080/01621459.1996.10476902
(1989). Structural equation modeling with latent variables. New York, NY: Wiley. https://doi.org/10.1002/9781118619179
(2002). Latent variables in psychology and the social sciences. Annual Review of Psychology, 53, 605–634. https://doi.org/10.1146/annurev.psych.53.100901.135239
(2010). Measurement error in nonlinear models: A modern perspective. Boca Raton, FL: Chapman and Hall/CRC.
(2000). A multitrait-multimethod model with minimal assumptions. Psychometrika, 65, 241–261. https://doi.org/10.1007/BF02294377
(2017). Anomalous results in G-factor models: Explanations and alternatives. Psychological Methods, 22, 541–562. https://doi.org/10.1037/met0000083
(2019). Beneficial for some or for everyone? Student characteristics as moderators of an autonomy-supportive intervention in the classroom. Journal of Educational Psychology, 111, 210–234. https://doi.org/10.1037/edu0000284
(2012). A comparison of four approaches to account for method effects in latent state-trait analyses. Psychological Methods, 17, 255–283. https://doi.org/10.1037/a0026977
(2010). Propensity score analysis. Thousand Oaks, CA: Sage.
(1985). Item response theory: Principles and applications. Boston, MA: Kluwer-Nijhoff.
(1990). Generalized additive models. London, UK: Chapman and Hall.
(2001). The elements of statistical learning: Data mining, inference, and prediction. New York, NY: Springer. https://doi.org/10.1007/978-0-387-21606-5
(1997). Instrumental variables: A study of implicit behavioral assumptions used in making program evaluations. Journal of Human Resources, 32, 441–462. https://doi.org/10.2307/146178
(1937). The bi-factor method. Psychometrika, 2, 41–54. https://doi.org/10.1007/BF02287965
(2015). Causal inference in statistics, social, and biomedical sciences. New York, NY: Cambridge University Press. https://doi.org/10.1017/cbo9781139025751
(2006). Demystifying double robustness: A comparison of alternative strategies for estimating a population mean for incomplete data. Statistical Science, 22, 523–539. https://doi.org/10.1214/07-STS227
(2016). Implementation and evaluation of a stress prevention program with teacher students. European Journal of Health Psychology, 24, 89–101. https://doi.org/10.1026/0943-8149/a000159
(1981). Linear panel analysis. New York, NY: Academic Press.
(2011). Effects of adult inpatient group psychotherapy on attachment characteristics: An observational study comparing routine care to an untreated comparison group. Psychotherapy Research, 22, 95–114. https://doi.org/10.1080/10503307.2011.626807
(2008). Probability theory – A comprehensive course. London, UK: Springer. https://doi.org/10.1007/978-1-84800-048-3
(2009). Estimation of average causal effects in quasi-experimental designs: Non-linear constraints in structural equation models (Unpublished doctoral dissertation). Jena, Germany: Friedrich-Schiller-University.
(1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
(1995).
Latent variable models of multitrait-multimethod data . In R. H. HoyleEd., Structural equation modeling. Concepts, issues, and applications (pp. 177–198). Thousands Oaks, CA: Sage.(2016). The EffectLiteR approach for analyzing average and conditional effects. Multivariate Behavioral Research, 51, 374–391. https://doi.org/10.1080/00273171.2016.1151334
(2014). Analyzing average and conditional effects with multigroup multilevel structural equation models. Frontiers in Psychology, 5, 304. https://doi.org/10.3389/fpsyg.2014.00304
(2019). Analysis of variance models with stochastic group weights. Multivariate Behavioral Research, 54, 542–554. https://doi.org/10.1080/00273171.2018.1548960
(2014). Theory and analysis of total, direct and indirect causal effects. Multivariate Behavioral Research, 49, 425–442. https://doi.org/10.1080/00273171.2014.931797
(2017). Effect analysis using non-linear structural equation mixture modeling. Structural Equation Modeling, 24, 556–570. https://doi.org/10.1080/10705511.2016.1273780
(1989). Generalized linear models (2nd ed.). Boca Raton, FL: Chapman & Hall/CRC.
(2015). Differential effects of reading trainings on reading processes: A comparison in Grade 2. Zeitschrift für Erziehungswissenschaft, 18, 489–512. https://doi.org/10.1007/s11618-015-0648-0
(2009). Causality: Models, reasoning, and inference (2nd ed.). Cambridge, UK: https://doi.org/10.1017/cbo9780511803161
(2008). Modelling method effects as individual causal effects. Journal of the Royal Statistical Society: Series A, 171, 41–63. https://doi.org/10.1111/j.1467-985X.2007.00517.x
Raudenbush, S. W.Bryk, A. S. (Eds.). (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
(2012). Propensity score analysis with fallible covariates: A note on a latent variable modeling approach. Educational and Psychological Measurement, 72, 715–733. https://doi.org/10.1177/0013164412440999
(1994). Estimation of regression coefficients when some regressors are not always observed. Journal of the American Statistical Association, 89, 846–866. https://doi.org/10.1080/01621459.1994.10476818
(1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55. https://doi.org/10.1093/biomet/70.1.41
(2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1–36. https://doi.org/10.18637/jss.v048.i02
(1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66, 688–701. https://doi.org/10.1037/h0037350
(2008). Average causal effects from observational studies: A practical guide and simulated example. Psychological Methods, 13, 279–313. https://doi.org/10.1037/a0014268
(2018). When does measurement error in covariates impact causal effect estimates? Analytic derivations of different scenarios and an empirical illustration. British Journal of Mathematical and Statistical Psychology, 72, 244–270. https://doi.org/10.1111/bmsp.12146
(2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
(2005). Analyzing individual and average causal effects via structural equation models. Methodology, 1, 39–54. https://doi.org/10.1027/1614-1881.1.1.39
(1997). Modeling true intraindividual change: True change as a latent variable. Methods of Psychological Research Online, 2, 21–33.
(1992). States and traits in psychological assessment. European Journal of Psychological Assessment, 8, 79–98. http://hdl.handle.net/10993/12805
(2000). Causal regression models II: Unconfoundedness and causal unbiasedness. Methods of Psychological Research Online, 5, 55–87.
(2014).
Causal inference on total, direct, and indirect effects . In A. C. MichalosEd., Encyclopedia of quality of life and well-being research (pp. 606–631). Dordrecht, The Netherlands: Springer. https://doi.org/10.1007/978-94-007-0753-5_295(2015). A theory of states and traits – revised. Annual Review of Clinical Psychology, 11, 71–98. https://doi.org/10.1146/annurev-clinpsy-032813-153719
(2008). EffectLite for Mplus: A program for the uni- and multivariate analysis of unconditional, conditional and average mean differences between groups [Computer software and manual]. Retrieved from http://www.statlite.com/
(2004). Matching methods for estimating causal effects using multiple control groups (Unpublished doctoral dissertation). Department of Statistics, Harvard University.
(1997).
Exploring the measurement invariance of psychological instruments: Applications in the substance use domain . In K. J. BryantM. WindleS. G. WestEds., The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281–324). Washington, DC: American Psychological Association. https://doi.org/10.1037/10222-009(2010). Econometric analysis of cross section and panel data (2nd ed.). Cambridge, MA: MIT.
Appendix A
EffectLiteR Code for Multistate Effect Model
mm <- ’
xi =~ c(1, 1)*y11 + c(1, 1)*y21
eta =~ c(1, 1)*y12 + c(1, 1)*y22
y11 ~ c(0, 0)*1
y21 ~ c(nu2, nu2)*1
y12 ~ c(0, 0)*1
y22 ~ c(nu2, nu2)*1
’
m1 <- effectLite(y=“eta”, x=“x”, z=“xi”, data=d, measurement=mm)
m1
## conditional effect for xi=3
newdata <- data.frame(xi=3)
elrPredict(m1, newdata=newdata)
Appendix B
EffectLiteR Code for True-Change Effect Model
mm <- ’
xi =~ c(1, 1)*y11 + c(1, 1)*y21 + c(1, 1)*y12 + c(1, 1)*y22
pi =~ c(1, 1)*y12 + c(1, 1)*y22
y11 ~ c(0, 0)*1
y21 ~ c(nu2, nu2)*1
y12 ~ c(0, 0)*1
y22 ~ c(nu2, nu2)*1
’
m1 <- effectLite(y=“pi”, x=“x”, z=“xi”, data=d, measurement=mm)
m1
Appendix C
EffectLiteR Code for Multistate Effect Model With Method Factor
mm <- ’
xi =~ c(1, 1, 1, 1)*y11 + c(1, 1, 1, 1)*y21
eta =~ c(1, 1, 1, 1)*y12 + c(1, 1, 1, 1)*y22
mf =~ c(1,1,1,1)*y21 + c(1,1,1,1)*y22
y11 ~ c(0, 0, 0, 0)*1
y21 ~ c(0, 0, 0, 0)*1
y12 ~ c(0, 0, 0, 0)*1
y22 ~ c(0, 0, 0, 0)*1
’
m1 <- effectLite(y=“eta”, x=“x”, k=“k”, z=c(“xi”,“mf”), data=d, measurement=mm, propscore=x ~ z1 + z2 + z3)
m1
Appendix D
EffectLiteR Code for Multitrait Effect Model With Method Factor
mm <- ’
xi1 =~ c(1, 1, 1, 1)*y11 + c(1, 1, 1, 1)*y21
xi2 =~ c(1, 1, 1, 1)*y12 + c(1, 1, 1, 1)*y22
eta1 =~ c(1, 1, 1, 1)*y13 + c(1, 1, 1, 1)*y23
eta2 =~ c(1, 1, 1, 1)*y14 + c(1, 1, 1, 1)*y24
xi =~ 1*xi1 + 1*xi2
eta =~ 1*eta1 + 1*eta2
mf =~ c(1, 1, 1, 1)*y21 + c(1, 1, 1, 1)*y22 + c(1, 1, 1, 1)*y23 + c(1, 1, 1, 1)*y24
y11 ~ c(0, 0, 0, 0)*1
y21 ~ c(0, 0, 0, 0)*1
y12 ~ c(0, 0, 0, 0)*1
y22 ~ c(0, 0, 0, 0)*1
y13 ~ c(0, 0, 0, 0)*1
y23 ~ c(0, 0, 0, 0)*1
y14 ~ c(0, 0, 0, 0)*1
y24 ~ c(0, 0, 0, 0)*1
’
m1 <- effectLite(y=“eta”, x=“x”, k=“k”, z=c(“xi”,“mf”), data=d, measurement=mm, propscore=x ~ z1 + z2 + z3)
m1
